{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 机器学习练习 2 - 逻辑回归"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "这个笔记包含了以Python为编程语言的Coursera上机器学习的第二次编程练习。请参考 [作业文件](ex2.pdf) 详细描述和方程。\n",
    "在这一次练习中，我们将要实现逻辑回归并且应用到一个分类任务。我们还将通过将正则化加入训练算法，来提高算法的鲁棒性，并用更复杂的情形来测试它。\n",
    "\n",
    "代码修改并注释：黄海广，haiguang2000@qq.com"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 逻辑回归"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "在训练的初始阶段，我们将要构建一个逻辑回归模型来预测，某个学生是否被大学录取。设想你是大学相关部分的管理者，想通过申请学生两次测试的评分，来决定他们是否被录取。现在你拥有之前申请学生的可以用于训练逻辑回归的训练样本集。对于每一个训练样本，你有他们两次测试的评分和最后是被录取的结果。为了完成这个预测任务，我们准备构建一个可以基于两次测试评分来评估录取可能性的分类模型。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "让我们从检查数据开始。"
   ]
  },
  {
   "cell_type": "code",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2024-08-07T02:04:37.679383Z",
     "start_time": "2024-08-07T02:04:37.042357Z"
    }
   },
   "source": [
    "import numpy as np\n",
    "import pandas as pd\n",
    "import matplotlib.pyplot as plt"
   ],
   "outputs": [],
   "execution_count": 1
  },
  {
   "cell_type": "code",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2024-08-07T02:04:37.725687Z",
     "start_time": "2024-08-07T02:04:37.703433Z"
    }
   },
   "source": [
    "path = 'ex2data1.txt'\n",
    "data = pd.read_csv(path, header=None, names=['Exam 1', 'Exam 2', 'Admitted'])\n",
    "data.head()"
   ],
   "outputs": [
    {
     "data": {
      "text/plain": [
       "      Exam 1     Exam 2  Admitted\n",
       "0  34.623660  78.024693         0\n",
       "1  30.286711  43.894998         0\n",
       "2  35.847409  72.902198         0\n",
       "3  60.182599  86.308552         1\n",
       "4  79.032736  75.344376         1"
      ],
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Exam 1</th>\n",
       "      <th>Exam 2</th>\n",
       "      <th>Admitted</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>34.623660</td>\n",
       "      <td>78.024693</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>30.286711</td>\n",
       "      <td>43.894998</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>35.847409</td>\n",
       "      <td>72.902198</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>60.182599</td>\n",
       "      <td>86.308552</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>79.032736</td>\n",
       "      <td>75.344376</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "execution_count": 2
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "让我们创建两个分数的散点图，并使用颜色编码来可视化，如果样本是正的（被接纳）或负的（未被接纳）。"
   ]
  },
  {
   "cell_type": "code",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2024-08-07T02:04:37.998244Z",
     "start_time": "2024-08-07T02:04:37.816478Z"
    }
   },
   "source": [
    "positive = data[data['Admitted'].isin([1])]\n",
    "negative = data[data['Admitted'].isin([0])]\n",
    "\n",
    "fig, ax = plt.subplots(figsize=(12,8))\n",
    "ax.scatter(positive['Exam 1'], positive['Exam 2'], s=50, c='b', marker='o', label='Admitted')\n",
    "ax.scatter(negative['Exam 1'], negative['Exam 2'], s=50, c='r', marker='x', label='Not Admitted')\n",
    "ax.legend()\n",
    "ax.set_xlabel('Exam 1 Score')\n",
    "ax.set_ylabel('Exam 2 Score')\n",
    "plt.show()"
   ],
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<Figure size 1200x800 with 1 Axes>"
      ],
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAA+0AAAKnCAYAAADz86ytAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjguNCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8fJSN1AAAACXBIWXMAAA9hAAAPYQGoP6dpAACAq0lEQVR4nO3de3xU1b3///dMhoThknhPSE2FKEGRoCitmnDJscFqvfAlbUWtUUT7LQdaBSwqtUUaKyjWgMoJnHrFeDymX3pSit+vCmiDuVTFCxqRn7ESdVQC1WIGZMyYZP/+2M2QSSbJJJnLnpnX8/GYxzB7ryQr7EDmvddan2UzDMMQAAAAAACwHHu0OwAAAAAAAAIjtAMAAAAAYFGEdgAAAAAALIrQDgAAAACARRHaAQAAAACwKEI7AAAAAAAWRWgHAAAAAMCiCO0AAAAAAFiUI9odsIL29nZ99tlnGjlypGw2W7S7AwAAAACIc4Zh6ODBg8rMzJTd3vN4OqFd0meffaasrKxodwMAAAAAkGBcLpdOPPHEHs8T2iWNHDlSkvmXlZqaGuXeAAAAAADindvtVlZWli+P9oTQLvmmxKemphLaAQAAAAAR09cSbQrRAQAAAABgUYR2AAAAAAAsitAOAAAAAIBFsaYdAAAAAIJgGIZaW1vV1tYW7a4gBiQlJcnhcAx6W3FCOwAAAAD0wev1au/evTp8+HC0u4IYMmzYMI0aNUrJyckD/hyEdgAAAADoRXt7uxobG5WUlKTMzEwlJycPevQU8c0wDHm9Xv3jH/9QY2Ojxo4dK7t9YKvTCe0AAAAA0Auv16v29nZlZWVp2LBh0e4OYoTT6dSQIUP00Ucfyev1aujQoQP6PBSiAwAAAIAgDHSkFIkrFD8z/NQBAAAAAGBRhHYAAAAAgM/y5ct15plnDupzfPjhh7LZbNq5c2dI+hSMgoICLVy4MGJfL1II7QAAAAAQ5+rq6pSUlKQLL7wwIl8vKytLe/fu1YQJEyRJVVVVstls+vLLL/3axWvQDqWohvaXXnpJl156qTIzM2Wz2fTnP//Z77xhGFq+fLkyMzPldDpVUFCgXbt2+bVpaWnRL37xCx133HEaPny4LrvsMn3yyScR/C4AAAAAIHgej7Rvn/kcKY8++qh+8YtfqKamRh9//HHYv15SUpIyMjLkcFD7fLCiGtq/+uornXHGGVq7dm3A86tWrVJpaanWrl2rHTt2KCMjQzNmzNDBgwd9bRYuXKjKyko9/fTTqqmp0aFDh3TJJZeora0tUt8GAAAAAPSppkYqKpJGjJAyMsznoiKptja8X/err77SH//4R/37v/+7LrnkEj3++ON+5++++26lp6dr5MiRuv766/X111/7nZ8zZ47+1//6X1qxYoXS09N11FFH6be//a1aW1u1ZMkSHXPMMTrxxBP16KOP+j6m8/T4Dz/8UP/2b/8mSTr66KNls9k0Z84czZkzR9u3b9f9998vm80mm82mDz/8UJL07rvv6gc/+IFGjBih9PR0FRcX6/PPP/f7nq655hqNGDFCo0aN0n333ReevzwLiGpov+iii/S73/1ORUVF3c4ZhqE1a9bo9ttvV1FRkSZMmKANGzbo8OHDeuqppyRJzc3NeuSRR3TfffepsLBQkyZN0pNPPqn6+npt27Yt0t8OAAAAAAS0bp00bZq0ebPU3m4ea283X0+dKq1fH76vXVFRoXHjxmncuHG6+uqr9dhjj8kwDEnSH//4R91xxx2666679Nprr2nUqFEqKyvr9jlefPFFffbZZ3rppZdUWlqq5cuX65JLLtHRRx+tV155RfPmzdO8efPkcrm6fWxWVpb+9Kc/SZLee+897d27V/fff7/uv/9+nXfeefrpT3+qvXv3au/evb5p9dOnT9eZZ56p1157Tc8995z27dunyy+/3Pc5lyxZor/+9a+qrKzUli1bVFVVpddffz1Mf4PRZdk17Y2NjWpqatIFF1zgO5aSkqLp06errq5OkvT666/rm2++8WuTmZmpCRMm+NoE0tLSIrfb7fcAAAAAgHCoqZEWLJAMQ2pt9T/X2moenz8/fCPujzzyiK6++mpJ0oUXXqhDhw7phRdekCStWbNGc+fO1Q033KBx48bpd7/7ncaPH9/tcxxzzDF64IEHNG7cOM2dO1fjxo3T4cOH9atf/Upjx47V0qVLlZycrNoA30RSUpKOOeYYSdIJJ5ygjIwMpaWlKS0tTcnJyRo2bJgyMjKUkZGhpKQkrVu3TmeddZZWrFihU089VZMmTdKjjz6qv/71r2poaNChQ4f0yCOP6Pe//71mzJih3NxcbdiwIW5nW1s2tDc1NUmS0tPT/Y6np6f7zjU1NSk5OVlHH310j20CWblype+HJC0tTVlZWSHuPQAAAACYSkulpKTe2yQlSatXh/5rv/fee3r11Vd1xRVXSJIcDodmz57tm8q+e/dunXfeeX4f0/W1JJ1++ul+e46np6crNze3U/+TdOyxx2r//v2D7vPrr7+uv/71rxoxYoTvceqpp0qSPvjgA33wwQfyer1+/TzmmGM0bty4QX9tK7J8VQCbzeb32jCMbse66qvN0qVLtXjxYt9rt9tNcAcAAAAQch6PtGnTkSnxPWltlSorzfZOZ+i+/iOPPKLW1lZ961vf8h0zDENDhgzRgQMHgv48Q4YM8Xtts9kCHmvv6xsNQnt7uy699FLdc8893c6NGjVK77///qC/Riyx7Eh7RkaGJHUbMd+/f79v9D0jI0Ner7fbD1vnNoGkpKQoNTXV7wEAAAAAoeZ29x3YO7S3m+1DpbW1VU888YTuu+8+7dy50/d46623dNJJJ+m//uu/dNppp+nll1/2+7iur0MhOTlZkrpNYU9OTu527KyzztKuXbs0evRonXLKKX6P4cOH65RTTtGQIUP8+nngwAE1NDSEvN9WYNnQPmbMGGVkZGjr1q2+Y16vV9u3b1deXp4k6eyzz9aQIUP82uzdu1fvvPOOrw0AAAAAREtqqmQPMnXZ7Wb7UHnmmWd04MABXX/99ZowYYLf40c/+pEeeeQR3XTTTXr00Uf16KOPqqGhQXfccUe3bbZD4aSTTpLNZtMzzzyjf/zjHzp06JAkafTo0XrllVf04Ycf6vPPP1d7e7sWLFigf/7zn7ryyiv16quvas+ePdqyZYvmzp2rtrY2jRgxQtdff72WLFmiF154Qe+8847mzJnjN30/nkT1uzp06JDvbo9kFp/buXOnPv74Y9lsNi1cuFArVqxQZWWl70IMGzZMV111lSQpLS1N119/vW6++Wa98MILevPNN3X11VcrNzdXhYWFUfzOIEnyegd3HgAAAIhxTqc0c6bU13blDoc0a1bop8YXFhYqLS2t27kf/vCH2rlzp8aOHatly5bp1ltv1dlnn62PPvpI//7v/x66TvzLt771Lf32t7/VbbfdpvT0dP385z+XJP3yl79UUlKSxo8fr+OPP14ff/yxMjMzVVtbq7a2Nn3/+9/XhAkTdNNNNyktLc0XzO+9915NmzZNl112mQoLCzVlyhSdffbZIe+3FdiMjlr/UVBVVeXbr6+za6+9Vo8//rgMw9Bvf/tb/ed//qcOHDigc845R//xH/+hCRMm+Np+/fXXWrJkiZ566il5PB5973vfU1lZWb/WqLvdbqWlpam5uZmp8qFSUSEtWyZt2yYFuhYul1RYKJWUSLNnR75/AAAAQJC+/vprNTY2asyYMRo6dGi/P76mxtzurbfkZbNJ1dVSfv4gOgrL6e1nJ9gcGtXQbhWE9hDzeqXcXKmhQcrOlqqq/IO7yyUVFEh79kg5OVJ9vfSvNS4AAACA1Qw2tEvmPuzz55tV4jtv++ZwSG1tUlmZNG9eiDoMywhFaI/PSf+IruRkc4Q9O9sM5gUFZlCX/AN7drbZjsAOAACAODdvnjmSPnPmkTXudrv5urqawI6eWX7LN8SorCxzhL0joBcUSOXlUnHxkcDedQQeAAAAiGP5+ebD4zGrxKemhnYNO+IToR3h0zW4dyzQIbADAAAggTmdhHUEj+nxCK+sLHOEvbPycgI7AAAAAASB0I7wcrnMKfGdFRcfWeMOAAAAAOgRoR3h07XoXG1t4OJ0AAAAAICACO0Ij66BvapKyssznwnuAAAL83ikffvMZwAAoo3QjtDzeqXCwsBV4juK03UE98JCsz0AAFFWUyMVFUkjRkgZGeZzUZE5UQwAgGghtCP0kpOlkhIpJydwlfiO4J6TY7Zjn/ZeMeIDAOG3bp00bZq0ebPU3m4ea283X0+dKq1fH93+AUAiKCgo0MKFCwf1OR5//HEdddRRIelPsGw2m/785z+H7fMT2hEes2dL9fU9V4nPyjLPz54d2X7FEEZ8ACAyamqkBQskw5BaW/3Ptbaax+fP5/9fALFpzpw5stlsuvvuu/2O//nPf5bNZuvX5xo9erTWrFkTdPsVK1YoKSmp29cOp9mzZ6uhocH3evny5TrzzDO7tQt30A4lQjvCp68RdEbYe8SIDwBETmmplJTUe5ukJGn16sj0B0Cc6mtJaBiXjA4dOlT33HOPDhw4ELavEchjjz2mW265RY8++mjEvqbT6dQJJ5wQsa8XCYR2wGIY8QGAyPF4pE2buv9/21Vrq1RZyVIlAANUUSHl5vZchNnlMs9XVITlyxcWFiojI0MrV67std2f/vQnnX766UpJSdHo0aN13333+c4VFBToo48+0qJFi2Sz2focpd++fbs8Ho9KSkr01Vdf6aWXXvI7/9VXX+maa67RiBEjNGrUKL+v1WH06NH63e9+52t30kknadOmTfrHP/6hmTNnasSIEcrNzdVrr73m+5jO0+Mff/xx/fa3v9Vbb73l6/Pjjz+u0aNHS5JmzZolm83mey1Jmzdv1tlnn62hQ4cqOztbv/3tb9Xa6ZfE+++/r2nTpmno0KEaP368tm7d2uvfQygQ2gGLYcQHACLH7T4yo6kv7e1mewDoF69XWrZMamgIvHtSx65LDQ1muzCMuCclJWnFihV68MEH9cknnwRs8/rrr+vyyy/XFVdcofr6ei1fvly/+c1v9Pjjj0uS/ud//kcnnniiSkpKtHfvXu3du7fXr/nII4/oyiuv1JAhQ3TllVfqkUce8Tu/ZMkS/fWvf1VlZaW2bNmiqqoqvf76690+z+rVq5Wfn68333xTF198sYqLi3XNNdfo6quv1htvvKFTTjlF11xzjQzD6Paxs2fP1s0336zTTz/d1+fZs2drx44dksyZAHv37vW9fv7553X11Vfrxhtv1Lvvvqv//M//1OOPP6677rpLktTe3q6ioiIlJSXp5Zdf1vr163Xrrbf2/pcfCgaM5uZmQ5LR3Nwc7a4gwR0+bBh2u2GY4+m9P+x2sz0AYOD4fxdAMDwej/Huu+8aHo9nYJ/g448NIzvb/M8kO9t83dvxELr22muNmTNnGoZhGOeee64xd+5cwzAMo7Ky0ugcB6+66ipjxowZfh+7ZMkSY/z48b7XJ510krF69eo+v2Zzc7MxbNgwY+fOnYZhGMabb75pDBs2zJe3Dh48aCQnJxtPP/2072O++OILw+l0GjfddJPf17v66qt9r/fu3WtIMn7zm9/4jv3tb38zJBl79+41DMMwHnvsMSMtLc13/o477jDOOOOMbn2UZFRWVvodmzp1qrFixQq/Y+Xl5caoUaMMwzCM559/3khKSjJcLpfv/LPPPhvwc3Xo7Wcn2BzKSDtgIYz4AAgGu0qEjtMpzZwpORy9t3M4pFmzzPYA0G9dtz0uKJDq6sznQNskh8k999yjDRs26N133+12bvfu3crPz/c7lp+fr/fff19tbW39+jpPPfWUsrOzdcYZZ0iSzjzzTGVnZ+vpp5+WJH3wwQfyer0677zzfB9zzDHHaNy4cd0+18SJE31/Tk9PlyTl5uZ2O7Z///5+9TGQ119/XSUlJRoxYoTv8dOf/lR79+7V4cOHtXv3bn3729/WiSee6PuYzt9DuBDaAQtJTZXsQf6rtNvN9gASB7tKhMfixVJf70fb2qRFiyLTHwweN7ZgSV2De35+RAO7JE2bNk3f//739atf/arbOcMwuq1TNwJMOQ/Go48+ql27dsnhcPgeu3bt8k2R78/nHTJkiO/PHf0LdKw92JGvXrS3t+u3v/2tdu7c6XvU19fr/fff19ChQwP2u78V+AeC0A5YCCM+QGIKJmCwq0T4TJkilZVJNlv3/38dDvN4WZn5/hrWxo0tWF5WllRe7n+svDwigb3D3Xffrc2bN6uurs7v+Pjx41VTU+N3rK6uTjk5OUr6V8Gl5OTkPkfd6+vr9dprr6mqqsov/L700kvasWOH3nnnHZ1yyikaMmSIXn75Zd/HHThwwG+rtlDpqc9Dhgzpdvyss87Se++9p1NOOaXbw263a/z48fr444/12Wef+T7mb3/7W8j73BWhHbAYRnyAxBFswGBXifCbN0+qrjZvnHbMeLLbzdfV1eZ5WBs3thATXC6puNj/WHFxz1XlwyA3N1c/+clP9OCDD/odv/nmm/XCCy/ozjvvVENDgzZs2KC1a9fql7/8pa/N6NGj9dJLL+nTTz/V559/HvDzP/LII/rud7+radOmacKECb7HlClTdN555+mRRx7RiBEjdP3112vJkiV64YUX9M4772jOnDmyBzvltB9Gjx6txsZG7dy5U59//rlaWlp8x1944QU1NTX5tsJbtmyZnnjiCS1fvly7du3S7t27VVFRoV//+teSzCr848aN0zXXXKO33npL1dXVuv3220Pe564I7YDFMOIDJIb+BAx2lYiM/Hxp40bp0CGpqcl83rjRGv/fMt27d9zYQkzoqBLfMSW+ttZ/jXsEg/udd97Zbar3WWedpT/+8Y96+umnNWHCBC1btkwlJSWaM2eOr01JSYk+/PBDnXzyyTr++OO7fV6v16snn3xSP/zhDwN+3R/+8Id68skn5fV6de+992ratGm67LLLVFhYqClTpujss88O6ffZ8TUvvPBC/du//ZuOP/54/fd//7ck6b777tPWrVuVlZWlSZMmSZK+//3v65lnntHWrVv1ne98R+eee65KS0t10kknSZLsdrsqKyvV0tKi7373u7rhhht8leXDyWYMdKFCHHG73UpLS1Nzc7NSWSQMi6itNd+AV1aab+TtdnNK/KJF1ngDCWDgamrMwN7bb2CbzRzhPesscwQ+mKV6drsZNFk6Ez9qasybNps2HfldMHOmdPPN/C7orKjIvOHVNbB35nCYf3cbN0auX4gfX3/9tRobGzVmzBgNHTq0/5+ga2DvWMPe03HEjd5+doLNoYy0AxZl5REfAIPTn5FzdpVIXEz3Do7HY97U6C2wS+b5ykpmKyAKvF6psDBwMO9anK6wMCz7tCO2EdoBi3M6pfR0Rs6AeNHfgDFkCLtKJCKmewePG1uwvORkqaREyskJPJLeEdxzcsx2ycnR6CUsjNAOAEAE9TdgfPMNu0okIuoYBI/tUhETZs+W6ut7nvqelWWenz07sv1CTCC0AwAQQQMJGOwqkViY7t0/bJeKmNHXCDoj7OgBoR0962s9DettAKDfBhIw2FUisTDdu/+4sQUgnhHaEVhFhZSb2/PWEy6Xeb6iIrL9AoA4MJCAwT7iiYPp3v3HjS1EChtvob9C8TNDaEd3Xq+0bJnU0BB4z8iOrSkaGsx2jLgDQL8MNGCwq0RiYLr3wHBjC+E0ZMgQSdLhw4ej3BPEmo6fmY6foYFgn3axT3tA7CUJAGFXW2sWEqusPLIH96xZ5gg7QTyx1dSY27319i7NZjPDKD8r3Xk85rKB1FRuaiB09u7dqy+//FInnHCChg0bJpvNFu0uwcIMw9Dhw4e1f/9+HXXUURo1alS3NsHmUEK7CO096hrQy8ul4mICOwCEGAEDgaxfb27rlpTkX5TO4TCXT5SVMXoMRJJhGGpqatKXX34Z7a4ghhx11FHKyMgIeJOH0N4PhPZedA7uHQjsAABEBLMxAOtpa2vTN998E+1uIAYMGTJESb3s30lo7wdCex/q6vzfGdTWSnl50esPAAAJhtkYABB/gs2hFKJD71wuc0p8Z8XFPVeVBwAAIed0SunpBHYASESEdvSs65r22lrzec+ewFXlAQAAAAAhRWhHYIGqxOflmc8EdwAAAACICEI7uvN6pcLCwFXis7L8g3thIfu0AwAAAECYENrRXXKyVFIi5eQErhLfEdxzcsx2ycnR6CUAAAAAxD2qx4vq8T3yensP5H2dBwAAAAAERPV4DF5fgZzADgAAAABhRWgHAAAAAMCiCO0AAAAAAFgUoR0AAAAAAIsitAMAAAAAYFGEdgAAAAAALIrQDgAAAACARRHaAQAAAACwKEI7AABACHk80r595jMAAINFaAcAAAiBmhqpqEgaMULKyDCfi4qk2tpo9wwAEMsI7QAAAIO0bp00bZq0ebPU3m4ea283X0+dKq1fH93+AQBiF6EdAABgEGpqpAULJMOQWlv9z7W2msfnz2fEHQAwMIR2AACAQSgtlZKSem+TlCStXh2Z/gAA4guhHQAAYIA8HmnTpu4j7F21tkqVlRSnAwD0H6EdAABggNzuI2vY+9LebrYHAKA/CO0AAAADlJoq2YN8N2W3m+0Re9jGD0A0EdoBAAAGyOmUZs6UHI7e2zkc0qxZZnvEDrbxA2AFhHYAAIBBWLxYamvrvU1bm7RoUWT6g9BgGz8AVkFoBwAAGIQpU6SyMslm6z7i7nCYx8vKpPz86PQP/cc2fgCshNAOAAAwSPPmSdXV5lT5jjXudrv5urraPI/YwTZ+AKzEZhiGEe1ORJvb7VZaWpqam5uVSoUYAAAwCB6PWSU+NZU17LHI4zHXrgezK4DdLh06xHUGMDDB5tA+yqYAAACgP5xOQlwsG8g2flxvAOHE9HgAAADgX9jGD4DVENoBAACAf2EbPwBWQ2gHAAAAOmEbPwBWQmgHAAAAOultG7+kJLbxAxBZhHYAAACgi45t/LoG87Y2aepUKTc3Ov0CkHgI7QAAAEAAb78tvfRS99H2ujozuK9fH51+AUgshHYAAACgi5oaacECyTCk1lb/c62t5vH586Xa2uj0D0DiILQDAABgQDwead8+8znelJaa69d7k5QkrV4dmf4ASFyEdgAAAPRLTY1UVCSNGCFlZJjPRUXxM+rs8UibNnUfYe+qtVWqrIzPmxYArIPQDgAAgKCtWydNmyZt3iy1t5vH2tvN1/GyztvtPvK99aW93WwPAOFCaAcAAEBQEmWdd2qqZA/yXbLdbrYHgHAhtAMAACAoibLO2+mUZs7sXjW+K4dDmjXLbA8A4WL50H7w4EEtXLhQJ510kpxOp/Ly8rRjxw7fecMwtHz5cmVmZsrpdKqgoEC7du2KYo8BAADiT6Kt81682NyTvTdtbdKiRZHpD4DEZfnQfsMNN2jr1q0qLy9XfX29LrjgAhUWFurTTz+VJK1atUqlpaVau3atduzYoYyMDM2YMUMHDx6Mcs8BAADiR6Kt854yRSork2y27iPuDod5vKxMys+PTv8AJA6bYRhGtDvRE4/Ho5EjR2rTpk26+OKLfcfPPPNMXXLJJbrzzjuVmZmphQsX6tZbb5UktbS0KD09Xffcc49+9rOfBfV13G630tLS1NzcrFQWJQEAAMjjMYN3aqo5/dvjMavEBxPc7Xbp0KH4mDZeW2tO96+sNL93u92cEr9oEYEdwOAEm0MtPdLe2tqqtrY2DR061O+40+lUTU2NGhsb1dTUpAsuuMB3LiUlRdOnT1ddXV2Pn7elpUVut9vvAQAAgJ63c3vjjcRc552fL23caN6EaGoynzduJLADiBxLh/aRI0fqvPPO05133qnPPvtMbW1tevLJJ/XKK69o7969ampqkiSlp6f7fVx6errvXCArV65UWlqa75GVlRXW7wMAACAW9LWdW3Z24q7zdjql9PT4uRkBIHZYOrRLUnl5uQzD0Le+9S2lpKTogQce0FVXXaWkTqVLbTab38cYhtHtWGdLly5Vc3Oz7+FyucLWfwCIJx6PtG9f7BeYAtBdMNu5lZZKN9/MOm8AiCTLh/aTTz5Z27dv16FDh+RyufTqq6/qm2++0ZgxY5SRkSFJ3UbV9+/f3230vbOUlBSlpqb6PQAAPetpumys78UM4Ihgt3NrbJSqq82p8h17mdvt5uvqamnevPD3FQASieVDe4fhw4dr1KhROnDggJ5//nnNnDnTF9y3bt3qa+f1erV9+3bl5eVFsbcAED/6mi67fn10+wdg8Pq7ndtZZ7HOGwAipY9SItH3/PPPyzAMjRs3Tn//+9+1ZMkSjRs3Ttddd51sNpsWLlyoFStWaOzYsRo7dqxWrFihYcOG6aqrrop21wEg5vU1XVaS5s+XcnN5sw7EsoFs5+Z0HnkAAMLH8qG9ublZS5cu1SeffKJjjjlGP/zhD3XXXXdpyJAhkqRbbrlFHo9H8+fP14EDB3TOOedoy5YtGjlyZJR7DgCxr2O6bG+jb0lJ5nZIhHYgdqWmmlPcg93OjZWFgL+uWyQCoWTpfdojhX3aAaC7RN2TGUhURUXmspfebtI5HOba9Y0bI9cvwMpqaswb3Js2mb8vO+o73HwzN7PRt7jYpx0AQonK5/0zkOmyAGLX4sWJu50bMBDUfEGkENoBxD0qnw9Mx3TZYDBdFoh9U6aY27WxnRvQt2C2SJw/n/caCA1CO4C4xl3wgXM6zSl+Xd+8d+VwSLNmMTUeiAfz5rGdGxCMYLdIXL06Mv1BfGNNu1jTDsSrmhozsPf2v5zNZr4RZeQoMP4OgcRFYS0gMGq+IFRY0w4g4XEXfPCYLgskLqdTSk8nbABdUfMFkUZoBxCXPB6zkmtvVZAl83xlJcXpesN0WQAAjqDmCyLN8vu0A8BADOQuOKNJPcvPNx9MlwUAJLqOmi/BbpHI70sMFiPtAOISd8HDg+myAACwRSIii9AOIC5R+RwAAPSXxyPt29f3sjlqviCSCO0A4hZ3wQEgeoINP4AV1NRIRUVmVfiMDPO5qKj3fdap+YJIIbQDiFvcBQeAyBtI+AGiad06c3vTzZuP1MNpbzdfT50qrV/f88fm50sbN5rbujU1mc8bN/LeAqFFaAcSVKKMgHAXHAAiZzDhB4iGmhppwQLJMLoXlWttNY/Pn9/3TSdqviCcCO1AgknEERDuggNA+IUq/ACRVFoqJSX13iYpSVq9OjL9AQIhtAMJJNFHQLgLDgDhQ/hBrPF4pE2bet+2TTLPV1bG/+xEWBehHUgQjIAAAMKF8INY5HYfGcToS3u72R6DlyhLNEOJ0A4kCEZAAADhQvhBLEpNPVLvpi92u9keA5eISzRDhdAOJABGQAAA4UT4QSxyOs3CtF13mOnK4ZBmzWJ53WAk+hLNwSK0AwmAERAAQDgRfhCrFi+W2tp6b9PWJi1aFJn+xCOWaA4eoR1IAIyAAJHHmj0kGsIPYtGUKVJZmWSzdb/p5HCYx8vK2HFmMFiiOXiEdiABMAICRA5r9pCoCD+IVfPmSdXV5nuljkEOu918XV1tnsfAsEQzNGyGYRjR7kS0ud1upaWlqbm5WakMMSJO1dSYa4l6+xdvs5m/nHhDBQzMunXmFMCkJP83KA6HOcJYVsabP8S/2lpzxKyy0lxyZbebN4QXLeL3C6zP4zGXCaamMogRCvv2mTewg9XUZG7PmyiCzaGEdhHakTjWrzfXDBEogNDjxhjgj/ADwOMxZ5wFU1vJbpcOHUqs/y+CzaFMjwcSCNO/gPBhzR4STV91G5xOc8Qskd6AA/DHEs3QYKRdjLQjMTECAoQOIwlIJDU15k2qTZuOTH+fOVO6+WZmkQDojploPWOkHUCvGAEBQodtFWMDFf0Hj72WAfQXRSoHj9AOAMAgsa2itVHRPzTYaxnAQLFEc3AI7QAADBJr9qyLkeHQoW4DgMHIz5c2bjSXiDU1mc8bNzLCHgzWtIs17QCAwWPNnvVwTUKHug0AEHqsaQcAIIJYs2c9jAyHDnUbACB6CO0AAIQIa/asw+Mxq5t3XXvdVWurVFlJcbq+ULcBAKKnj9V3AACgP/LzzQfbKkbXQEaGuU49/9x21G3YvLn3GyEOh9mOv0sACB1G2gEACAO2VYwuRob7J5gK+4sXS21tvX+etjZp0aLw9hUAEg2hHQAAxB0q+gcv2Ar71G0AgOggtAMAgLjEyHDf+rv3OnUbACDyCO0AACAuMTLct4FU2GevZQCILEI7AACIW4wM92ywFfap2wAAkUH1eAAAENeo6B8YFfYBIDYQ2gEAQEJwOgmdnXVU2A8muFNhHwCih+nxAAAACYgK+wAQGwjtAAAACYoK+wBgfYR2AACABEWFfQCwPkI7AABAAqPCPgBYG4XoAAAAEhwV9gHAugjtAAAAkESFfQCwIqbHAwAAAABgUYR2AAAAAAAsitAOAAAAAIBFEdoBAAAAALAoQjsAAAAAABZFaAcAAAAAwKII7QAAAAAAWBShHQAAAAAAiyK0AwAAAABgUYR2AAAAAAAsitAOAAAAAIBFEdoBAAAAALAoQjsAAAAAABZFaAcAAAAAwKII7QAAAAAAWBShHQAAAAAAiyK0AwAAAABgUYR2AAAAAAAsitAOAAAC8nikffvMZwAAEB2EdgAA4KemRioqkkaMkDIyzOeiIqm2Nto9AwAg8RDaAQCAz7p10rRp0ubNUnu7eay93Xw9daq0fn10+wcAQKIhtAMAAEnmCPuCBZJhSK2t/udaW83j8+cz4g4AQCQR2gEAgCSptFRKSuq9TVKStHp1ZPoDAAAI7QAAQGaxuU2buo+wd9XaKlVWUpwOAIBIIbQDAAC53UfWsPelvd1sDwAAwo/QDgAAlJoq2YN8V2C3m+0BAED4EdoBAICcTmnmTMnh6L2dwyHNmmW2BwAA4Wfp0N7a2qpf//rXGjNmjJxOp7Kzs1VSUqL2TvP3DMPQ8uXLlZmZKafTqYKCAu3atSuKvQYAIDYtXiy1tfXepq1NWrQoMv0BAAAWD+333HOP1q9fr7Vr12r37t1atWqV7r33Xj344IO+NqtWrVJpaanWrl2rHTt2KCMjQzNmzNDBgwej2HMAAGLPlClSWZlks3UfcXc4zONlZVJ+fnT6BwBAIrJ0aP/b3/6mmTNn6uKLL9bo0aP1ox/9SBdccIFee+01SeYo+5o1a3T77berqKhIEyZM0IYNG3T48GE99dRTUe49AACxZ948qbranCrfscbdbjdfV1eb5wEAQORYOrRPmTJFL7zwghoaGiRJb731lmpqavSDH/xAktTY2KimpiZdcMEFvo9JSUnR9OnTVVdXF5U+AwAQ6/LzpY0bpUOHpKYm83njRkbYAQCIhj7KzUTXrbfequbmZp166qlKSkpSW1ub7rrrLl155ZWSpKamJklSenq638elp6fro48+6vHztrS0qKWlxffazb41AAB043RScA4AgGiz9Eh7RUWFnnzyST311FN64403tGHDBv3+97/Xhg0b/NrZbDa/14ZhdDvW2cqVK5WWluZ7ZGVlhaX/AAAAAAAMhqVD+5IlS3TbbbfpiiuuUG5uroqLi7Vo0SKtXLlSkpSRkSHpyIh7h/3793cbfe9s6dKlam5u9j1cLlf4vgkAAAAAAAbI0qH98OHDstv9u5iUlOTb8m3MmDHKyMjQ1q1bfee9Xq+2b9+uvLy8Hj9vSkqKUlNT/R4AAAAAAFiNpde0X3rppbrrrrv07W9/W6effrrefPNNlZaWau7cuZLMafELFy7UihUrNHbsWI0dO1YrVqzQsGHDdNVVV0W594CFeL1ScvLAzwMAAACICkuPtD/44IP60Y9+pPnz5+u0007TL3/5S/3sZz/TnXfe6Wtzyy23aOHChZo/f74mT56sTz/9VFu2bNHIkSOj2HPAQioqpNxcqadlIC6Xeb6iIrL9AgAAANAnm2EYRrQ7EW1ut1tpaWlqbm5mqjzii9drBvKGBik7W6qqkjoXXnS5pIICac8eKSdHqq9nxB0AAACIgGBzqKVH2gEMUnKytG2bGdj37DEDeseIe+fAnp1ttiOwA0DUeDzSvn3mMwAAHQjtQLzLyjJH2DsH97o6/8DedQQeABAxNTVSUZE0YoSUkWE+FxVJtbXR7hkAwAqYHi+mxyNBdB5Z70BgB4CoWrdOWrBASkqSWluPHHc4pLY2qaxMmjcvev0DAIQP0+MB+MvKksrL/Y+VlxPYASBKamrMwG4Y/oFdMl8bhjR/PiPuAJDoCO2IHq93cOfRPy6XVFzsf6y4uOeq8gASFmurI6O01Bxh701SkrR6dWT6AwCwJkI7ooNtyCKra9G52trAxekAJDTWVkeOxyNt2tR9hL2r1lapspIbKACQyAjtiDyvV1q2zNyGLFBg7AiYDQ1mO0bcB6drYK+qkvLyuhenI7gDCW3dOmnaNGnzZqm93TzW3m6+njpVWr8+uv2LN273kb/nvrS3m+0BAImJ0I7IYxuyyPF6pcLCwFXiu1aVLyzkBgmQoFhbHXmpqZI9yHdhdrvZHgCQmAjtiA62IYuM5GSppETKyQn899lxHXJyzHbcIAESEmurI8/plGbONKvE98bhkGbNMtsDABITW76JLd+iim3IIsPr7T2Q93UeQNzyeMy168FM1bbbpUOHCJChUlNjLkno7Z2YzSZVV0v5+ZHrFwAgMtjyDbGBbcgio69ATmAHEhZrq6NnyhRzH3abrfuIu8NhHi8rI7ADQKIjtCO62IYMAKKKtdXRNW+eOZI+c+aR62C3m6+rq83zANATtuhMDIR2RA/bkAFA1LG2Ovry86WNG82lB01N5vPGjYywA+gZW3QmFkI7ooNtyADAMhYvltraem/T1iYtWhSZ/iQqp1NKT+fGCIDesUVn4iG0I/LYhgwALIW11QAQG9iiMzER2hF5bEMGAJbD2moAsD626ExMbPkmtnyLGrYhAwBL8njMKvGpqUzVBiKBf3MIBlt0xh+2fIP1sQ0ZAFgSa6uByKCYGPqDLToTF6EdAAAAiDCKiaG/2KIzcRHaAQAAgAiimBgGgi06ExehHYiUvqrgUyUfAICEQDExDBRbdCYmQjsQCRUVUm5uz/vOu1zm+YqKyPYLQI88HmnfPvMZAELF45E2beo+wt5Va6tUWcn/QfDHFp2JidAOhJvXKy1bJjU0SAUF3YO7y2Ueb2gw2zHinnAIh9ZCYSgA4UQxMQwWW3QmHkI7EG7JydK2bVJ2trRnj39w7wjse/aY57dto2p+AiEcWg+FoQCEG8XEEAr5+dLGjea2bk1N5vPGjYywxytCOxAJWVlSVZV/cK+r8w/sVVVmOyQEwqH1UBgKQCRQTAyhxBadiYHQDkRK1+Cen09gT1CEQ2uiMBSASKGYGID+ILQDkZSVJZWX+x8rLyewJxjCofVQGAqIf1aqH0IxMQD9QWhHZLDdmcnlkoqL/Y8VF/dcVR5xh3BoTRSGAuKXVeuHUEwMQLAI7Qg/tjszdS06V1sbuDgd4hrh0JooDAXEJ6vXD6GYGIBgENoRXmx3Zuoa2KuqpLy87sXpCO5xj3BoTRSGAuJPLNUPoZgYgN4Q2hFebHdm3ogoLAxcdK5rcbrCwvi9cQFJhEMrozAUEF+oHwIgXhDaEX6Jvt1ZcrJUUiLl5AT+Pjv+fnJyzHbxeOMCfgiH1kRhKCB+UD8EQDyxGYZhRLsT0eZ2u5WWlqbm5malMhc1fDqPrHeI98DemdfbeyDv6zziyvr15rTMpCT/N5UOhxnYy8ooQhQttbXmyFtlpbn21W43Zz0sWkRgB2LFvn1m0blgNTWZ09MBIJKCzaGEdhHaI6quzv9db22tubYbSECEQ2vzeMxCgKmpLFMAYo3HY1aJD6bwp91uFoDj3zmASAs2h/axqhIIoZ62O0uUkXagi/x880E4tCank+sBxKqO+iGbN/c+Rd7hMNvxbx2AlbGmHZHBdmdAj6gaDAChN5j6IR6POcWete4ArIDQjvBjuzMAABBhAykuWVMjFRWZU+szMsznoiJrbAsHIHER2hFebHcGAACiZN48qbranAJv/9e7XrvdfF1d7V/wc906ado0c0p9x1r49nbz9dSpZgFRAIgGCtGJQnRhV1EhLVtm7sMeaO26y2UG9pISafbsyPcPAADEvd7qh9TUmIG9t3fFNpsZ9CkUCiBUqB7fD4T2CGC7MwAAYFFFRcEXrdu4MXL9AhDfgs2hTI9HZPQVyAnsAAAgCjweadOm3gO7ZJ6vrKQ4HYDII7QDAAAgYbndwe3nLpnt3O7w9gcAuiK0AwAAIGGlph4pUtcXu91sDwCRRGgHAABAwnI6zbXqXbeF68rhkGbN6l7EDgDCjdAOAEAEeDzSvn2shwWsaPFiqa2t9zZtbdKiRZHpDwB0RmgHACCMamrMytQjRkgZGeZzUZFUWxvtngHoMGWKVFZmbuvWdcTd4TCPl5Wx3RuA6CC0IzZ5vYM7DwARsG6duffz5s1HCl21t5uvp06V1q+Pbv8AHDFvnrkP+8yZR9a42+3m6+pq8zwARAOhHbGnokLKzZVcrsDnXS7zfEVFZPsFAJ3U1EgLFkiG0X0rqdZW8/j8+YFH3JlKD0RHfr65D/uhQ1JTk/m8cSMj7ACii9CO2OL1SsuWSQ0NUkFB9+DucpnHGxrMdoy4A4iS0lIpKan3NklJ0urVR14zlR6wBqdTSk+n6BwAayC0I7YkJ0vbtknZ2dKePf7BvSOw79ljnt+2zWwPABHm8UibNnUfYe+qtVWqrDTbM5UeAAAEQmhH7MnKkqqq/IN7XZ1/YK+qMtsBQBS43UeCd1/a26WtWwc+lR4AAPiLt2VmhHbEpq7BPT+fwA7AMlJTjxSy6ovdLj3ySP+n0gMAAH/xusyM0I7YlZUllZf7HysvJ7ADiDqn06w43XXrqK4cDumyy6RnnunfVHoAAOAvnpeZEdoRu1wuqbjY/1hxcc9V5QEgghYvltraem/T1ibNndu/qfRu9+D7BgBAPBnMji2xgNCO2NS16FxtbeDidAAQJVOmSGVlks3WfcTd4TCPl5VJhYX9m0qfmhr6vgIAEMsGsmNLLCG0I/Z0DexVVVJeXvfidAR3AFE2b55UXW1Ole8I5na7+bq62jzfn6n0s2axBRUAAJ0NZMeWWGMzDMOIdieize12Ky0tTc3NzUplCMPavF4pN9fchz1Q0bnOgT4nR6qvZ9s3AJbg8ZhT21NTuwfvmhpzHV5vv5FtNjPo5+eHt58AAMSSffvMonPBamqS0tPD15/+CDaHMtKO2JKcLJWUmIE8UJX4jqryOTlmOwI7AItwOs03CYFGyoOdSk9gBwDAX393bInFMVpCO2LP7NnmCHpPVeKzsszzs2dHtl8AMAjBTKUHAAD+EmGZGdPjxfR4AIC19DaVHgAA+IvVZWZMjwdikdc7uPMA4kJvU+kBAIC/eF9mRmgHrKKiwiyy11PVe5fLPF9REdl+AQAAABYXz8vMmB4vpsfDAqiKDwAAAIRErCwzY3o8EEuSk6Vt2wLvM991X/pt2wjsAAAAQA/ibZkZoR2wio7t6joH97o6/8AeaJs7AAAAAHGrj8L4ACKqI7h3BPWOahkEdgAAACAhMdIOWE1WllRe7n+svJzADgCIex6PtG+f+QwAMBHaAatxuaTiYv9jxcU9V5UHACDG1dRIRUXSiBFSRob5XFQk1dZGu2cAEH2EdsBKuhadq60NXJwOAIA4sW6dNG2atHmz1N5uHmtvN19PnSqtXx/d/gFAtBHaAavoGtirqqS8vO7F6QjuAIA4UVMjLVggGYbU2up/rrXVPD5/PiPuABIboR2wAq9XKiwMXCW+a1X5wkKzPQCINcCIbaWlUlJS722SkqTVqyPTHwCwIsuH9tGjR8tms3V7LFiwQJJkGIaWL1+uzMxMOZ1OFRQUaNeuXVHuNdBPyclSSYmUkxO4SnxHcM/JMduxTzuQ8FgDjFCJ1o0fj0fatKn7CHtXra1SZSU3ptAzbl4i3lk+tO/YsUN79+71PbZu3SpJ+vGPfyxJWrVqlUpLS7V27Vrt2LFDGRkZmjFjhg4ePBjNbgP9N3u2VF/fc5X4rCzz/OzZke0XAMthDTBCIdo3ftzuIz+/fWlvN9sDnUX7ZxiIlAGF9vLycuXn5yszM1MfffSRJGnNmjXatGlTSDsnSccff7wyMjJ8j2eeeUYnn3yypk+fLsMwtGbNGt1+++0qKirShAkTtGHDBh0+fFhPPfVUyPsChF1fI+iMsAMJjzXACAUr3PhJTZXsQb4TtdvN9kAHK/wMA5HS79C+bt06LV68WD/4wQ/05Zdfqq2tTZJ01FFHac2aNaHunx+v16snn3xSc+fOlc1mU2Njo5qamnTBBRf42qSkpGj69Omqq6sLa18AwKqYJhjfWAOMwbLKjR+nU5o5U3I4em/ncEizZpntAck6P8NApPQ7tD/44IN66KGHdPvttyup07uGyZMnq76+PqSd6+rPf/6zvvzyS82ZM0eS1NTUJElKT0/3a5eenu47F0hLS4vcbrffAwBiHdME4x9rgBEKVrrxs3ix9K/xnx61tUmLFoW/L4gdVvoZBiKh36G9sbFRkyZN6nY8JSVFX331VUg61ZNHHnlEF110kTIzM/2O22w2v9eGYXQ71tnKlSuVlpbme2T1tIYYAGIE0wQTA2uAMVhWu/EzZYpUVibZbN1H3B0O83hZmZSfH95+IHZY7WcYiIR+h/YxY8Zo586d3Y4/++yzGj9+fCj6FNBHH32kbdu26YYbbvAdy8jIkKRuo+r79+/vNvre2dKlS9Xc3Ox7uNj3GkAMY5pg4mANMAbLijd+5s2TqqvNqfIdP992u/m6uto8D3Sw4s8wEG59rCLqbsmSJVqwYIG+/vprGYahV199Vf/93/+tlStX6uGHHw5HHyVJjz32mE444QRdfPHFvmNjxoxRRkaGtm7d6hv993q92r59u+65554eP1dKSopSUlLC1lcAiKSOaYK9jTp0TBNktCq2dawB3ry59+vtcJjtWAOMrjpu/AQTeiJ54yc/33x4PGbISk3t+ec3mDaIX1b9GQbCqd8j7dddd53uuOMO3XLLLTp8+LCuuuoqrV+/Xvfff7+uuOKKcPRR7e3teuyxx3TttdfK0WnulM1m08KFC7VixQpVVlbqnXfe0Zw5czRs2DBdddVVYekLAFgJ0wQTD2uAMRhWL/7mdErp6YG/LnU7IFn/Zxg9o1DuwPUrtLe2tmrDhg269NJL9dFHH2n//v1qamqSy+XS9ddfH64+atu2bfr44481d+7cbuduueUWLVy4UPPnz9fkyZP16aefasuWLRo5cmTY+gMAVsE0wcTDGmAMVize+KFuBzqLxZ/hRBauG26JdBPAZhiG0Z8PGDZsmHbv3q2TTjopXH2KOLfbrbS0NDU3NyuVOTQAYojHY/7yC3aa4KFDjDrEi9pac8lDZaV5/e12c1Rp0SICO/q2fr1Z66Lr0hqHwww7ZWXWWUteU2MG9t7esdps5vp3fvYTRyz9DCeydevMujuhvE41NebSwE2bjvz+mzlTuvnm2Ps/INgc2u/p8eecc47efPPNQXUOABAaTBNMXPn50saN5o2YpibzeePG2HvDguiIpeJvbO+FQGLpZzhRhaNQbqLOuun3SPv/+T//R7fddpsWLVqks88+W8OHD/c7P3HixJB2MBIYaQcQyxiFAjAYVi7sxmwiBMPKP8OJrKgo+MKpGzf2/fni8f1OsDm036HdHmCvGZvN5tsbva2vBSYWRGgHEOuYJgggHu3bZ66BDVZTk1nIDkB0heOGW6hvAlhBsDm031u+NTY2DqpjAIDQmzdPys3tvsZ55kzWOAOIXWzvBcSmgRTK7S20d+yW09fn7LxbTjzNuuh3aI+nAnQAEE/6s88xAMSCjrodwY6u8X8eYA2hvuEW6psAsabfhegk6YMPPtAvfvELFRYWasaMGbrxxhv1wQcfhLpvAIAB6G2fYwCINWzvBcSeUBfK7bgJEIx4nHXT79D+/PPPa/z48Xr11Vc1ceJETZgwQa+88opOP/10bd26NRx9BAAAQIKaMsWsy2GzdQ8ADod5vKyMZUCA1YTyhlui75bT70J0kyZN0ve//33dfffdfsdvu+02bdmyRW+88UZIOxgJFKIDAACwttra7nU7Zs2ibgdgZaEslEv1+H4YOnSo6uvrNXbsWL/jDQ0Nmjhxor7++uuB9TiKCO0AAACxgbodQGwJ5Q23eNstJ2zV448//njt3LmzW2jfuXOnTjjhhP73FAAAAAiS00lYB2JJKAvlJupuOf0O7T/96U/1v//3/9aePXuUl5cnm82mmpoa3XPPPbr55pvD0UcAAAAAQAwL1Q23RNwtp9/T4w3D0Jo1a3Tffffps88+kyRlZmZqyZIluvHGG2Wz2cLS0XBiejwAAAAAIJLCtqa9s4MHD0qSRo4cOdBPYQmEdgAAAABAJIVtTXtjY6NaW1s1duxYv7D+/vvva8iQIRo9evSAOgwAAAAAAPz1e5/2OXPmqK6urtvxV155RXPmzAlFnwAAAAAAgAYQ2t98803lByjLd+6552rnzp2h6BMAAAAAANAAQrvNZvOtZe+sublZbW1tIekUAABATzwead8+8xkAgHjX79A+depUrVy50i+gt7W1aeXKlZoyZUpIOwcgDni9gzsPAP9SUyMVFUkjRkgZGeZzUZFUWxvtngEAED79Du2rVq3Siy++qHHjxum6667Tddddp3Hjxumll17SvffeG44+AohVFRVSbq7kcgU+73KZ5ysqItsvADFn3Tpp2jRp82apvd081t5uvp46VVq/Prr9AwAgXAa05dtnn32mtWvX6q233pLT6dTEiRP185//XMccc0w4+hh2bPkGhIHXawbyhgYpO1uqqpKyso6cd7mkggJpzx4pJ0eqr5eSk6PVWwAWVlNjBvbe3rHYbFJ1tRSg7A4AAJYUkX3a4wWhHQiTzsG8c3Dv6TgABFBUZI6ot7b23MbhkGbOlDZujFy/AAAYjGBzaNDT4//5z3/qk08+8Tu2a9cuXXfddbr88sv11FNPDby3AOJTVpYZyLOzzYBeUCDV1RHYAQTN45E2beo9sEvm+cpKitMBAOJP0KF9wYIFKi0t9b3ev3+/pk6dqh07dqilpUVz5sxReXl5WDoJIIZ1De75+QR2AEFzu4+sYe9Le7vZHgACYecJxKqgQ/vLL7+syy67zPf6iSee0DHHHKOdO3dq06ZNWrFihf7jP/4jLJ0EEOOysqSuN/XKywnsAPqUmirZg3y3Yreb7QGgM3aeQKwLOrQ3NTVpzJgxvtcvvviiZs2aJYfDIUm67LLL9P7774e+hwBin8slFRf7Hysu7rmqPAD8i9NprlX/19uNHjkc0qxZZnsA6JBoO08wmyA+BR3aU1NT9eWXX/pev/rqqzr33HN9r202m1paWkLaOQBxoGvRudpa/zXuBHcAfVi8WGpr671NW5u0aFFk+gMgNtTUSAsWmDtPdK2L0dpqHp8/Pz5G3JlNEN+CDu3f/e539cADD6i9vV0bN27UwYMHdf755/vONzQ0KIuprgA6C1QlPi+ve3E6gjuAXkyZIpWVmdu6dR1xdzjM42VlbPcGwF9pqZSU1HubpCRp9erI9CdcEm02QSIKesu3nTt3qrCwUAcPHlRra6t+9atf6c477/SdLy4u1vDhw7U+Bn8q2PINCAP2aQcQYrW15pvrykrzDandbk6JX7SIwA7An8djjjYHU8jSbpcOHYrN5TU1NWZg7y3R2WxSdTX/T1pRsDm0jxViR5x55pnavXu36urqlJGRoXPOOcfv/BVXXKHx48cPvMcA4ktyslRSIi1bJm3b1r3oXEdV+cJCsx2BHUAf8vPNh8djVolPTY3NN9kAwm8gO0/E4v8nHbMJetsWs2M2AaE9dgU90h7PGGnHoHm9vYfOvs7HM/5u0AWBCwAQbokw0p4I32O8CzaHBr2mHUAPKirMaeA9rct2uczzFRWR7ZdV9BXICewJgyI5AIBISYSdJwYymwCxidAODIbXa07/bmgIXFCtY912Q4PZzuuNRi+BqKNIDgAg0uJ954nUVHMEPRh2u9kesYnQDgxGcrK5XjtQJfSuldO3bWNUGQkpkbbcAQBYR7zvPJEIswlgIrQDg9VRUK1zcK+r677VGVsiIkElypY7AADrmTfPrJw+c+aRUWm73XxdXW2ej2XxPpsAJgrRiUJ0CJHOI+sdCOxIcBTJAQBYRbwWQl2/3pyx1rWKvMNhBvaysti/ORGvwlKIrqysTIWFhbr88sv14osv+p37/PPPlZ2dPbDeAvEgK0sqL/c/Vl5OYEdCo0gOAMAqnE4pPT2+ArsU/7MJ0I/Q/sADD2jJkiU69dRTlZKSoh/84AdauXKl73xbW5s++uijsHQSiAkul1Rc7H+suLjnqvJAAqBIDgAA4ZefL23caM5Ya2oynzdujN31+vAXdGj/z//8Tz300ENau3atysvL9de//lVr1qzRsmXLwtk/IDZ0LTpXWxu4OB2QYCiSAwBA5MTrbIJEF3Rob2xsVF5enu/1eeedpxdffFF/+MMftHTp0rB0DogJXQN7VZWUl9e9OB3BHQmKIjkAAAADF3RoP+644+TqEjpOP/10vfjii3rssce0ZMmSkHcOsDyvVyosDFwlvmtV+cJC9mlHQor3LXcAAADCKejQPmXKFP3pT3/qdnz8+PF64YUX9Nxzz4W0Y0BMSE6WSkqknJzAVeI7gntOjtmOfdqRoCiSAwAAMDBBb/n29ttv6/XXX9d1110X8PyuXbu0ceNG3XHHHSHtYCSw5RsGzevtPZD3dR5IIPG65Q4AAEB/BJtD2addhHYAAAAAQGSFZZ92AAAAAAAQOYR2AIgHfRU5pAgiAABATCK0A0Csq6iQcnN73lbQ5TLPV1REtl8AAAAYNEI7AMQyr1datkxqaJAKCroHd5fLPN7QYLZjxB0AACCmENoBIJYlJ0vbtknZ2dKePf7BvSOw79ljnt+2jV0MAAAAYky/Q/sXX3yhBQsWaPz48TruuON0zDHH+D0AABGWlSVVVfkH97o6/8BeVWW2AwAAQExx9PcDrr76an3wwQe6/vrrlZ6eLpvNFo5+AQD6oyO4dwT1/HzzOIEdAAAgpvU7tNfU1KimpkZnnHFGOPoDABiorCypvPxIYJfM1wR2AACAmNXv6fGnnnqqPB5POPoCABgMl0sqLvY/Vlzcc1V5AAAAWF6/Q3tZWZluv/12bd++XV988YXcbrffAwAQBV2LztXWBi5OBwAAgJjS7+nxRx11lJqbm3X++ef7HTcMQzabTW1tbSHrHAAgCF0De8ca9s5r3AsKWNsOAAAQg/od2n/yk58oOTlZTz31FIXoACDavF6psDBwlfiuwb2wUKqvZ9s3AACAGNLv0P7OO+/ozTff1Lhx48LRHwBAfyQnSyUl0rJl5j7sXUfSO4J7YaHZjsAOAAAQU/od2idPniyXy0VoBwCrmD1bmjWr50CelcUIOwAAQIzqd2j/xS9+oZtuuklLlixRbm6uhgwZ4nd+4sSJIescACBIfQVyAjsAAEBMshmGYfTnA+z27gXnbTZbTBeic7vdSktLU3Nzs1JTU6PdHQAAAABAnAs2h/Z7pL2xsXFQHQMAAAAAAMHpd2g/6aSTwtEPAAAAAADQRb9De4d3331XH3/8sbxer9/xyy67bNCdAgAAic3jkdxuKTVVcjqj3RsAgNUk0u+Jfof2PXv2aNasWaqvr/etZZfk2689Fte0AwAAa6ipkUpLpU2bpPZ2yW6XZs6Ubr5Zys+Pdu8AANGWiL8nuleV68NNN92kMWPGaN++fRo2bJh27dqll156SZMnT1ZVVVUYuggA6JcuM6D6fR6IknXrpGnTpM2bzTdikvm8ebM0daq0fn10+wcAiK5E/T3R79D+t7/9TSUlJTr++ONlt9tlt9s1ZcoUrVy5UjfeeGM4+ggACFZFhZSbK7lcgc+7XOb5iorI9gvoQ02NtGCBZBhSa6v/udZW8/j8+VJtbXT6BwCIrkT+PdHv0N7W1qYRI0ZIko477jh99tlnkswCde+9915oewcACJ7XKy1bJjU0SAUF3YO7y2Ueb2gw2zHiDgspLZWSknpvk5QkrV4dmf4AAKwlkX9P9Du0T5gwQW+//bYk6ZxzztGqVatUW1urkpISZWdnh7yDiFNM3wVCLzlZ2rZNys6W9uzxD+4dgX3PHvP8tm1me8ACPB5zbWLXkZOuWlulykqzPQAgcST674l+h/Zf//rXav/XAoLf/e53+uijjzR16lT9v//3//TAAw+EvIOIQ0zfBcInK0uqqvIP7nV1/oG9qspsB1iE231kbWJf2tvN9gCAxJHovydsRkf590H45z//qaOPPtpXQT7WuN1upaWlqbm5WampqdHuTnzzes1A3tAQODx0Hg3MyZHq6xkNBAai87+lDgR2WJTHI40YEdwbMrtdOnQo/rf3AQAcEa+/J4LNof0ead+3b1+3Y8ccc4xsNptv2jzQI6bvApGRlSWVl/sfKy8nsMOSnE5zux5HHxvROhzSrFmx8UYMABA6if57ot+hPTc3V3/5y1+6Hf/973+vc845JySdQpxj+i4Qfi6XVFzsf6y4uOdlKUCULV4stbX13qatTVq0KDL9AQBYSyL/nuh3aL/11ls1e/ZszZs3Tx6PR59++qnOP/983XvvvaoIwxrkTz/9VFdffbWOPfZYDRs2TGeeeaZef/1133nDMLR8+XJlZmbK6XSqoKBAu3btCnk/EGJdg3t+PoEd8cEKRRa7zlqprQ08uwWwkClTpLIyyWbrPpLicJjHy8rMXxcAgMSTyL8n+h3ab775Zr388suqra3VxIkTNXHiRDmdTr399tu67LLLQtq5AwcOKD8/X0OGDNGzzz6rd999V/fdd5+OOuooX5tVq1aptLRUa9eu1Y4dO5SRkaEZM2bo4MGDIe0LwoDpu4g3Viiy2DWwV1VJeXndZ7cQ3GFB8+ZJ1dXmFEj7v96h2O3m6+pq8zwAIHEl6u+JARWiO3jwoH7605/qT3/6kyTp4Ycf1rXXXhvyzt12222qra1VdXV1wPOGYSgzM1MLFy7UrbfeKklqaWlRenq67rnnHv3sZz8L6utQiC5KKJSFeGKFIotW6AMQIh6PWf03NTX+1iYCAAYvHn5PhK0QXccI+9///ne9/fbbWrdunX7xi1/o8ssv14EDBwbV6a7+8pe/aPLkyfrxj3+sE044QZMmTdJDDz3kO9/Y2KimpiZdcMEFvmMpKSmaPn266urqQtoXhBjTdxFvrFBkMTlZKikxA3mgm18dy1Jycsx2cRjYPR5p37742581ETmdUnp67L4RAwCEVyL9nuh3aD///PM1e/Zs/e1vf9Npp52mG264QW+++aY++eQT5ebmhrRze/bs0bp16zR27Fg9//zzmjdvnm688UY98cQTkqSmpiZJUnp6ut/Hpaen+84F0tLSIrfb7fdABDF9F/HKCkUWZ882R9B7+hpZWeb52bPD14coqKmRiorM7WAyMsznoiLzfiAAAEAs63do37Jli+6++24NGTLEd+zkk09WTU1N0NPRg9Xe3q6zzjpLK1as0KRJk/Szn/1MP/3pT7Vu3Tq/dl33hzcMo9c941euXKm0tDTfI4up2JHj9UqFhYEDTNfAU1gYmaJdQChZochiXyPocTbCvm6dNG2atHnzkf1b29vN11OnSuvXR7d/AAAAg9Hv0D59+vTAn8hu129+85tBd6izUaNGafz48X7HTjvtNH388ceSpIyMDEnqNqq+f//+bqPvnS1dulTNzc2+h4sR3chh+i4SAUUWI6amRlqwQDIMqbXV/1xrq3l8/nxG3AEAQOwKOrT/4Ac/UHNzs+/1XXfdpS+//NL3+osvvugWsAcrPz9f7733nt+xhoYGnXTSSZKkMWPGKCMjQ1u3bvWd93q92r59u/Ly8nr8vCkpKUpNTfV7IIISdPouEgh7pEdMaamUlNR7m6QkafXqyPQHAAAg1IIO7c8//7xaWlp8r++55x7985//9L1ubW3tFrAHa9GiRXr55Ze1YsUK/f3vf9dTTz2lP/zhD1qwYIEkc1r8woULtWLFClVWVuqdd97RnDlzNGzYMF111VUh7QtCLMGm7yKBUGQxYjweadOm7iPsXbW2SpWVFKcDAACxydF3E1PXneEGsFNcv33nO99RZWWlli5dqpKSEo0ZM0Zr1qzRT37yE1+bW265RR6PR/Pnz9eBAwd0zjnnaMuWLRo5cmTY+wcAfgIVWexY8tFxvKCAbQ1DxO0+soa9L+3tZvtEqDALAADiS9D7tNvtdjU1NemEE06QJI0cOVJvvfWWsrOzJUn79u1TZmam2trawtfbMGGfdgCDxh7pEefxmFXigwnudrt06BChHQAAWEfI92m32WzdKrL3VqEdABIKRRYjzumUZs6UHH3MGXM4pFmzCOwAACA29Wt6/Jw5c5SSkiJJ+vrrrzVv3jwNHz5ckvzWuwNAQpo920yHPQXyjiKLBPaQWbxY+vOfe2/T1iYtWhSR7gAAAIRc0KH92muv9Xt99dVXd2tzzTXXDL5HABDLKLIYUVOmSGVl5rZuSUn+RekcDjOwl5VJ+fnR6yMAAMBgBL2mPZ6xph0AYlttrbmtW2WlucbdbjcnPSxaRGAHAADWFGwODXqkHQAAq8rPNx8ej1klPjWVNewAACA+ENoBAHHD6SSsAwCA+BJ09XgAAAAAABBZhHYAAAAAACyK0A4AAAAAgEUR2gEAAAAAsChCOwAAAAAAFkVoBwAAAADAogjtAAAAALrxeKR9+8xnANFDaAcAAADgU1MjFRVJI0ZIGRnmc1GRVFsb7Z4BiYnQDgAAAECStG6dNG2atHmz1N5uHmtvN19PnSqtXx/d/gGJiNAOAAAAQDU10oIFkmFIra3+51pbzePz5zPiDkQaoR0AAACASkulpKTe2yQlSatXR6Y/AEyEdgAAACDBeTzSpk3dR9i7am2VKispTgdEEqEdQOR5vYM7DwAAejSQqu9u95E17H1pbzfbA4gMQjuAyKqokHJzJZcr8HmXyzxfURHZfgEAEOMGU/U9NVWyB5kM7HazPYDIILQDiByvV1q2TGpokAoKugd3l8s83tBgtmPEHQCAoAy26rvTKc2cKTkcvbdzOKRZs8z2ACKD0A4gcpKTpW3bpOxsac8e/+DeEdj37DHPb9tmtgcAAL0KVdX3xYultrbe27S1SYsWDa6/APqH0A4gsrKypKoq/+BeV+cf2KuqzHYAAKBPoar6PmWKVFYm2WzdR9wdDvN4WZmUnz+4/gLoH5thGEa0OxFtbrdbaWlpam5uVioLdIDI6Dyy3oHADgBAv3g85tr1YIrI2e3SoUN9T22vrTUDfmWl+XntdnNK/KJFBHYglILNoX2sWgGAMMnKksrL/X/7l5cT2AEA6IeBVH3vK7Tn55sPj8dsn5rKGnYgmpgeDyA6XC6puNj/WHFxz1XlAQBAN+Gs+u50SunpBHYg2gjtACKva9G52trAxekAAECvqPoOxD9CO4DI6hrYq6qkvLzuxekI7gAABIWq70B8I7QDiByvVyosDFwlvmtV+cJC9mkHACAIVH0H4huhHUDkJCdLJSVSTk7gKvEdwT0nx2zHPu0AAARl3jyputqcKt+xxt1uN19XV5vnAcQmtnwTW74BEef19h7I+zoPAAB6RNV3IDaw5RsA6+orkBPYAQAYMKeTsA7EE6bHAwAAAABgUYR2AAAAAAAsitAOAAAAAIBFEdoBAAAAALAoQjsAAAAAABZFaAcAAAAAwKII7QAAAAAAWBShHQAAAAAAiyK0AwAAAABgUYR2AAAAAAAsitAOAAAAAIBFEdoBAAAAALAoQjsAAAAAABZFaAcAAAAAwKII7QAAAAAAWBShHQAAJDyPR9q3z3wGAMBKCO0AACBh1dRIRUXSiBFSRob5XFQk1dZGu2cAAJgI7QAAICGtWydNmyZt3iy1t5vH2tvN11OnSuvXR7d/AABIhHYAAJCAamqkBQskw5BaW/3Ptbaax+fPZ8QdABB9hHYAwOB4vYM7D0RBaamUlNR7m6QkafXqyPQHAICeENoBAANXUSHl5kouV+DzLpd5vqIisv0CeuHxSJs2dR9h76q1VaqspDgdACC6CO0AgIHxeqVly6SGBqmgoHtwd7nM4w0NZjtG3EODmQ2D5nYfWcPel/Z2sz0AANFCaAeAgSA4ScnJ0rZtUna2tGePf3DvCOx79pjnt20z22NwmNkQEqmpkj3Id0B2u9keAIBoIbQDQH8RnI7IypKqqvyDe12df2CvqjLbYXCY2RAyTqc0c6bkcPTezuGQZs0y2wMAEC2EdgDoD4JTd12De34+gT0cmNkQUosXS21tvbdpa5MWLYpMfwAA6AmhHQD6g+AUWFaWVF7uf6y8nMAeasxsCJkpU6SyMslm6z7i7nCYx8vKzHtQAABEE6EdAPqL4NSdyyUVF/sfKy7ueQkBBo6ZDSEzb55UXW1Ole9Y4263m6+rq83zAABEm80wDCPanYg2t9uttLQ0NTc3K5VqMwCC1XlkvUMiBqeuMwzKy83ATpAMr7o6/2Hg2lopLy96/YlxHo9ZJT41lTXsAIDICDaHMtIOAAPFlPDugb2qygyOXWciMOIeWsxsCDmnU0pPJ7ADAKyH0A4AA5XowcnrlQoLA4+od53CXViYGEX5IqHrjZLaWm6QAAAQxwjtADAQBCezyF5JiZSTE3gKfEdwz8kx2yVKUb5wYmYDAAAJhzXtYk07gH4KFJyysno+Hu+83t4DeV/nERyvV8rNNbcTDPTz1fnnLydHqq/n7x0AAAtjTTsAhANTwrvrKxgSHEODmQ0AACQkRtrFSDuAfqqokJYtM/dhDzSS7nKZgb2kRJo9O/L9Q3xjZgMAAHEh2BxKaBehHcAAEJwAAAAwCEyPB4BwYko4AAAAIoDQDgCAVfVVEyERaiYAAJDgCO0AAFhRRYVZLb6n7dtcLvN8RUVk+wUAACKK0A4AgNV4vWaxw4aGwPuud2zv1tBgtmPEHQCAuGXp0L58+XLZbDa/R0ZGhu+8YRhavny5MjMz5XQ6VVBQoF27dkWxxwDQB6Y7IxjJyebuBB3bB3YO7p33Y8/ONttRQwEAgLhl6dAuSaeffrr27t3re9TX1/vOrVq1SqWlpVq7dq127NihjIwMzZgxQwcPHoxijwGgB0x3Rn907LveObjX1fkH9kD7tQMAgLhi+dDucDiUkZHhexx//PGSzFH2NWvW6Pbbb1dRUZEmTJigDRs26PDhw3rqqaei3OsQYlQOiA9Md8ZAdA3u+fkEdgAAEozlQ/v777+vzMxMjRkzRldccYX27NkjSWpsbFRTU5MuuOACX9uUlBRNnz5ddXV10epuaDEqB8QPpjtjoLKypPJy/2Pl5QR2AAAShKVD+znnnKMnnnhCzz//vB566CE1NTUpLy9PX3zxhZqamiRJ6enpfh+Tnp7uO9eTlpYWud1uv4flMCoHxB+mO2MgXC6puNj/WHFxzzd0AQBAXLF0aL/ooov0wx/+ULm5uSosLNT//b//V5K0YcMGXxubzeb3MYZhdDvW1cqVK5WWluZ7ZFnxDTKjckB8Yroz+qPr//e1tYF/LwAAgLhl6dDe1fDhw5Wbm6v333/fV0W+66j6/v37u42+d7V06VI1Nzf7Hi6rvulhVA6IT0x3RjC6BvaqKikvr/vvBav+DgMAACERU6G9paVFu3fv1qhRozRmzBhlZGRo69atvvNer1fbt29XXl5er58nJSVFqampfg/LYlQOiD9Md0ZfvF6psDDw//ddfy8UFrJECgCAOGbp0P7LX/5S27dvV2Njo1555RX96Ec/ktvt1rXXXiubzaaFCxdqxYoVqqys1DvvvKM5c+Zo2LBhuuqqq6Ld9dBiVA6IH51HT8eM6X26M0EscSUnSyUlUk5O4Bu0HcE9J8dsxxIpAADiliPaHejNJ598oiuvvFKff/65jj/+eJ177rl6+eWXddJJJ0mSbrnlFnk8Hs2fP18HDhzQOeecoy1btmjkyJFR7nmI9TQqx0g7EFs6B/YTTjCPdYSvjuMFBeZryRxBLSmRZs+OSncRZbNnS7Nm9RzIs7Kk+noCOwAAcc5mGIYR7U5Em9vtVlpampqbm603Vb7rmsbycjOwM0UeiC1er7lFY0ODOcIuSY2NR/4dS/7/1g3DPJ+TQzADAACIQ8HmUEtPj094FCEC4kfn6c7bt5uPzv+OpSP/tg8dOhLo2R0CAAAgoRHarYoiRED8mT3bHDXPygq8O4TLZY6w79/PTBoAQFA8HmnfPvMZQHwitFsVRYiA+NT532qg3SE6T5knsAMAelBTIxUVSSNGSBkZ5nNRkVnfFEB8YU27LL6m3evtPZD3dR6A9dXVmYG9Q22tuRQGAIAA1q2TFiyQkpKk1tYjxx0Oqa1NKiuT5s2LXv8ABIc17fGir0BOYAdiG3u2AwD6oabGDOyG4R/YJfO1YUjz5zPiDsQTQjsAREvXYpO97dkOAICk0lJzhL03SUnS6tWR6Q+A8CO0A0A0sDsEAKCfPB5p06buI+xdtbZKlZUUpwPiBaEdACKN3SEAwA8V0IPjdkvt7cG1bW832wOIfYR2AIg0docAAElUQO+v1FTJHuS7d7vdbA8g9lE9XhavHg8gfrE7BIAERgX0gSkqkjZv7n2KvMMhzZwpbdwYuX4B6D+qxwOA1bE7BIAERQX0gVu82Lyp0Zu2NmnRosj0B0D4EdoBAAAQUVRAH7gpU8xZCDabOaLemcNhHi8rk/Lzo9M/AKFHaIe19VWAiwJdAADEFCqgD968eVJ1tTkFvmONu91uvq6uZlkBEG8I7bCuigopN7fnLa9cLvN8RUVk+wVEEzeyAMQ4KqCHRn6+uWb90CGpqcl83riREXYgHhHaYU1er7RsmdTQEHiv6o49rhsazHYEFSQCbmQBiANUQA8tp1NKTzefAcQnQjusKTlZ2rbtyF7VnYN7R2Dv2ON62zYKdiH+cSML6BkzUGKK02lO4+66Hrsrh0OaNYswCgCEdlhXx17VnYN7XZ1/YA+0xzUQj7iRBQTGDJSI8HikfftCt76cCugAEDxCO6yta3DPzyewI3FxIwvwxwyUsKupMfcFHzFCysgwn4uKBr8VGxXQASB4hHZYX1aWVF7uf6y8nGCCxMSNLOAIZqCE1bp10rRp0ubNRwrHtbebr6dOldavH9znpwI6AATHZhiGEe1ORJvb7VZaWpqam5uVSrUT6+n8xqsDAQWJrq7OfwiqtlbKy4tef4Bo6hrQy8ul4mJuaA1CTY0Z2Ht7l2izmeE6FKPhHo9ZJT41lTXsABJHsDmUkXZYW9c3YrW1gUdUgETicpmBpLPiYv49IHExAyXkSkulpKTe2yQlSatXh+brUQEdAHpGaId1dQ3sVVXmSGLXNb0EFSQSbmQBgbGUKmQ8HmnTJqm1tfd2ra1SZWXoitMBAAIjtMOavF6psDDwSEnXEZXCQooLIf4E+pnuHNjHjOFGFtAZM1BCxu0+soa9L+3tZnsAQPgQ2mFNyclSSYmUkxN4amNHcM/JMdtRXAjxJNAWVp1vZH372+Zi0ro68xw3spDomIESUqmpRwrD9cVuN9sDsK5Qb9mIyCO0w7pmz5bq63ue2piVZZ6fPTuy/QLCqactrDpuZI0ZY77es8d/CytuZCFRsZQq5JxOs4J7163YunI4pFmzWIcOWFW4tmxE5BHaYW19BQ+CCeJNb1tYdVSH//jjwFtYcSMLiYalVGGzeLHU1tZ7m7Y2adGiyPQHQP+Ee8tGRBahHQCspmvYKCgwp8IXFEiNjb1XxOZGFhIJS6nCZsoUqazMXInTdcTd4TCPl5WFZrs3INLifbp4TY20YIG5ZWPXgpKtrebx+fMZcY8lhHYAsCK2sAKCw1KqsJk3z9yHfebMI2vc7XbzdXW1eR6IJYkyXTzSWzYi/GyGYRjR7kS0BbupPQBEXF2d/1BWbe2RafIAECEej1klPjWVNeyITevWmaPPSUn+o88Oh7nUo6wsPm5EeTzmzYhgdoCw26VDh/g3HU3B5lBG2gHAqtjCCoBFOJ1Sejpv7hGbEmm6OFs2xidCOwBYEVtYAYAlxPv650SQSNPF2bIxPhHaAcBq2MIKAKIuUdY/xzuPR9q0qfsIe1etrVJlZezfnGHLxvhEaAcAK2ELKwCIOrbLih+JOF2cLRvjD6EdAKyELawAIKoSaf1zIkjE6eJs2Rh/CO0AYDVsYQUAUZNI658TQaJOF2fLxvjClm9iy7eI8Xp7HxXs6zwAAEAYsV1WfKqpMZc79JZ6bDYzzMbj6DNbNloXW77BWioqpNzcngtnuVzm+YqKyPYLAADgXxJx/XMiSPTp4mzZGPsI7Qg/r1datkxqaAhc8bqjUnZDg9mOwloAACAKEnH9c6JgujhiGaE9lvQVZq0adpOTpW3bAm9V1XVrq23bmCIPAACiIlHXPyeK/Hxp40ZzWUNTk/m8cWP8jrAjfhDaY0WsTy/vulVVQYFUV9d9L+qeCm8BAABEANtlxT+miyPWENpjQbxML+8a3PPzCewAAMBSEn39MwDrIbTHgniaXp6VJZWX+x8rLyewAwAAy4jX9c8ej7Rvn/kMIHYQ2mNFvEwvd7mk4mL/Y8XFPU/7BwAAiIJ4Wv9cUyMVFZnb2WVkmM9FRVJtbbR7Zk3c3IDVENpjSTSml4ey+F3XWQG1tYFnDwAAAFhErK9/XrfO3KN88+Yj29m1t5uvp06V1q+Pbv+shJsbsCpCe6yJ5PTyUBa/6xrYq6qkvLzuswcI7gAAACFRUyMtWCAZhtTa6n+utdU8Pn8+oVTi5gasjdAeayI1vTyUxe+8XqmwMPCsgK6zBwoLrVtIDwAAIIaUlkpJSb23SUqSVq+OTH+sipsbsDpCeyyJ5PTyUBa/S06WSkqknJzA0/g7gntOjtnOyoX0AADREcrlWkAC8HikTZu6h9CuWlulysrEXr/NzQ1YHaE9VkRjenkoi9/Nni3V1/fcNivLPD97duj6DwCID6FcrgVYQQRuQrndR6Z596W93WyfiLi5gVhAaI8F0ZxeHsrid32NoDPCDgDoKpTLtQAriNBNqNTUI9vV9cVuN9snIm5uIBYQ2mNBtKeXs7c6AAwMU7oHL5TLtYBoi+BNKKfT3Ffe4ei9ncMhzZoVu9XxB4ubG4gFhPZYEc3p5eytDgD9x5Tu0Anlci0gmiJ8E2rxYqmtrfc2bW3SokWD+jIxjZsbiAWE9lgSjenl7K0OAP3HlO7QC+VyLSCaIngTasoUqaxMstm6h1KHwzxeVmb+c0pk3NyA1RHa0TP2VgeAgWFKd3iwXAvxIoI3oebNk6qrzdHkjmngdrv5urraPJ/ouLkBq7MZhmFEuxPR5na7lZaWpubmZqWyUMXk9ZrTNhsaAv8C6fymMyfHnJrPm04A8Nc1oJeXm8uLGCEemM5/nx34e0Qsq6vzT4K1teYASZh4PGYhtdRUpnkHUltrbutWWWkWnbPbzSnxixYR2BEeweZQQrsI7T2qqDCnbW7bFvjNkMtlVqsvKWGrNgDoCUEzNLgBgnjD/w2Wxc0NRAqhvR8I7b3wensfQe/rPAAg4qNpcSfQcq2srJ6PA1bHTSgACj6HsqYdvWNvdQAYHHbgGByv15zVFSjMdF0XXFhIUT9YHzWDAPQToR0AgHBhB47BS042l2Hl5AQefewI7jk5ZjtuJg9OXzc9uCkyONyEAjAAhHYAAMKB0bTQmT3bLHja03ThrCzzPPVVBqeiwixC29PPpMtlnq+oiGy/4gk3oQAMAGvaxZp2AECIsQMHYg0/s5FFzSAAYk07AADRw2gaYk1ysrlbTKBZIF1njWzbxs/sYFEzCEA/MNIuRtoBAGHCaFpw+HuyDqqaA0DEMNIOAEC0MZrWN9ZRW0vXYmj5+QR2AIgyQjsAAIgOr1datsxcRx2oKF/HqG9Dg9mOStqRkZVljrB3Vl5OYAeAKCG0AwCA6GAdtTW5XOaU+M6Ki9npAACihNAOAACip+t07IICqa6u+3Z5jPJGRtebJbW1bFEYTn3NHmF2CQAR2gEAQLSxjtoaugb2qiopL6/7TRWCe2hQzwFAkAjtAAAg+lhHHV1er1RYGPhmSdebKoWFjAAPFvUcAPQDoR0AAEQf66ijKzlZKimRcnICz27oCO45OWY76gsMDvUcAPQDoR0AAEQX66itYfZsqb6+59kNWVnm+dmzI9uveEU9BwBBIrQDAIDoYR21tfQ1osuIb2hRzwFAEGIqtK9cuVI2m00LFy70HTMMQ8uXL1dmZqacTqcKCgq0a9eu6HUSAAAEh3XUAPUcAPQpZkL7jh079Ic//EETJ070O75q1SqVlpZq7dq12rFjhzIyMjRjxgwdPHgwSj0FAABBYR01QD0HAH2KidB+6NAh/eQnP9FDDz2ko48+2nfcMAytWbNGt99+u4qKijRhwgRt2LBBhw8f1lNPPRXFHgMAgKCwjhqJjHoOAIIQE6F9wYIFuvjii1VYWOh3vLGxUU1NTbrgggt8x1JSUjR9+nTV1dVFupsAAGAgWEeNREQ9BwBBckS7A315+umn9cYbb2jHjh3dzjU1NUmS0tPT/Y6np6fro48+6vFztrS0qKWlxffa7XaHqLcAAABAH4Kp59AR6AsLzdkm3LwCEpalR9pdLpduuukmPfnkkxo6dGiP7Ww2m99rwzC6Hets5cqVSktL8z2yKPQBANbRV7ExipEBiHXUcwDQD5YO7a+//rr279+vs88+Ww6HQw6HQ9u3b9cDDzwgh8PhG2HvGHHvsH///m6j750tXbpUzc3NvoeLaUcAYA0VFVJubs/TQV0u83xFRWT7BQChRj0HAEGydGj/3ve+p/r6eu3cudP3mDx5sn7yk59o586dys7OVkZGhrZu3er7GK/Xq+3btysvL6/Hz5uSkqLU1FS/BwAgyrxeadkyqaEh8DrOjvWfDQ1mO0bcAcQ66jkACIKl17SPHDlSEyZM8Ds2fPhwHXvssb7jCxcu1IoVKzR27FiNHTtWK1as0LBhw3TVVVdFo8sAgIFKTpa2bTuyjrOg4Mi00a4Fm7Zt480sACA+eL29/07r6zzinqVH2oNxyy23aOHChZo/f74mT56sTz/9VFu2bNHIkSOj3TUAQH91rOPsXDm5rq57hWVqkQAA4gHLwhAEm2EYRrQ7EW1ut1tpaWlqbm5mqjwAWEHnkfUOBHYAQDzxes1A3tAQ+Hdc59+FOTnsIhCHgs2hMT/SDgCIQ1lZUnm5/7HycgI7ACB+dCwL6zy7rGPEnWVh6ITQDgCwHpdLKi72P1Zc3PP0QQAAYhHLwhAEQjsAwFq6ji7U1gYehQAAIB50De75+dEJ7H3tysKuLVFDaAcAWEfXwF5VJeXldR+FILgDAOJJtJeFURDP0gjtAABr8HqlwsLAowtdRyEKC7njDwCIH9FcFub1SsuWmQXxAt0Y77ih3tBgtuP3b8QR2gEA1pCcLJWUmBVyA00H7AjuOTlmOwryAADiQbSXhVEQz/LY8k1s+QYAluL19v6GoK/zAADEikDLwrKyej4eyb6Ul5uj/RTEC5tgcyihXYR2AAAAABFmxX3aO3/NDgT2sGGfdgAAAACwKisuC4t2QTwExEi7GGkHAAAAECVWWhbGSHtEMdIOAAAAAFbXVyCPRmCPRkE89IjQDgAAAACJLFDhu7w8/+1WCe5RQ2gHAAAAgETl9UqFhYGrxHesq+8I7oWF7NMeBYR2AAAAAEhUViyIBz8UohOF6AAAAAAkOCsVxEsQFKIDAAAAAATHKgXx0A2hHQAAAAAAiyK0AwAAAABgUYR2AAAAAAAsitAOAAAAAIBFEdoBAAAAALAoQjsAAAAAABZFaAcAAAAAwKII7QAAAAAAWBShHQAQGV7v4M4DAAAkIEI7ACD8Kiqk3FzJ5Qp83uUyz1dURLZfAAAAFkdoBwCEl9crLVsmNTRIBQXdg7vLZR5vaDDbMeIOAADgQ2gHAIRXcrK0bZuUnS3t2eMf3DsC+5495vlt28z2AAAAkERoBwBEQlaWVFXlH9zr6vwDe1WV2Q4AAAA+jmh3AACQIDqCe0dQz883jxPYAQAAesRIOwAgcrKypPJy/2Pl5QR2AACAHhDaAQCR43JJxcX+x4qLe64qDwAAkOAI7QCAyOhadK62NnBxOgAAAPgQ2gEA4dc1sFdVSXl53YvTEdwBAAD8ENoBAOHl9UqFhYGrxHetKl9YyD7tAAAAnRDaAQDhlZwslZRIOTmBq8R3BPecHLMd+7QDAAD42AzDMKLdiWhzu91KS0tTc3OzUlNTo90dAIhPXm/vgbyv8wAAAHEk2BzKSDsAIDL6CuQEdgAAgG4I7QAAAAAAWBShHQAAAAAAiyK0AwAAAABgUYR2AAAAAAAsitAOAAAAAIBFEdoBAAAAALAoQjsAAAAAABZFaAcAAAAAwKII7QAAAAAAWBShHQAAAAAAiyK0AwAAAABgUYR2AAAAAAAsitAOAAAAAIBFEdoBAAAAALAoQjsAAAAAABZFaAcAAAAAwKII7QAAAAAAWBShHQAAAAAAiyK0AwAAAABgUYR2AAAAAAAsitAOAAAAAIBFEdoBAAAAALAoQjsAAAAAABZFaAcAAAAAwKII7QAAAAAAWBShHQAAAAAAiyK0AwAAAABgUYR2AAAAdOf1Du48ACAkCO0AAADwV1Eh5eZKLlfg8y6Xeb6iIrL9AoAERGgHAADAEV6vtGyZ1NAgFRR0D+4ul3m8ocFsx4g7AIQVoR0AAABHJCdL27ZJ2dnSnj3+wb0jsO/ZY57fts1sDwAIG0I7AAAA/GVlSVVV/sG9rs4/sFdVme0AAGHliHYHAAAAYEEdwb0jqOfnm8cJ7AAQUZYeaV+3bp0mTpyo1NRUpaam6rzzztOzzz7rO28YhpYvX67MzEw5nU4VFBRo165dUewxAABAHMnKksrL/Y+VlxPYASCCLB3aTzzxRN1999167bXX9Nprr+n888/XzJkzfcF81apVKi0t1dq1a7Vjxw5lZGRoxowZOnjwYJR7DgAAEAdcLqm42P9YcXHPVeUBACFnMwzDiHYn+uOYY47Rvffeq7lz5yozM1MLFy7UrbfeKklqaWlRenq67rnnHv3sZz8L+nO63W6lpaWpublZqamp4eo6AABA7OhadK683AzsrGkHgJAINodaeqS9s7a2Nj399NP66quvdN5556mxsVFNTU264IILfG1SUlI0ffp01dXVRbGnAAAAMa5rYK+qkvLyuhenY8QdAMLO8qG9vr5eI0aMUEpKiubNm6fKykqNHz9eTU1NkqT09HS/9unp6b5zPWlpaZHb7fZ7AAAAQOa+64WFgUfUu1aVLyxkn3YACDPLh/Zx48Zp586devnll/Xv//7vuvbaa/Xuu+/6zttsNr/2hmF0O9bVypUrlZaW5ntkMbULAADAlJwslZRIOTmBp8B3BPecHLMd+7QDQFjF3Jr2wsJCnXzyybr11lt18skn64033tCkSZN852fOnKmjjjpKGzZs6PFztLS0qKWlxffa7XYrKyuLNe0AAAAdvN7eA3lf5wEAvYq7Ne0dDMNQS0uLxowZo4yMDG3dutV3zuv1avv27crLy+v1c6SkpPi2ket4AAAAoJO+AjmBHQAiwhHtDvTmV7/6lS666CJlZWXp4MGDevrpp1VVVaXnnntONptNCxcu1IoVKzR27FiNHTtWK1as0LBhw3TVVVdFu+sAAAAAAAyapUP7vn37VFxcrL179yotLU0TJ07Uc889pxkzZkiSbrnlFnk8Hs2fP18HDhzQOeecoy1btmjkyJFR7jkAAAAAAIMXc2vaw4F92gEAAAAAkRS3a9oBAAAAAEgUhHYAAAAAACyK0A4AAAAAgEUR2gEAAAAAsChCOwAAAAAAFkVoBwAAAADAogjtAAAAAABYFKEdAAAAAACLIrQDAAAAAGBRhHYAAAAAACyK0A4AAAAAgEUR2gEAAAAAsChCOwAAAAAAFkVoBwAAAADAogjtAAAAAABYFKEdAAAAAACLckS7A1ZgGIYkye12R7knAAAAAIBE0JE/O/JoTwjtkg4ePChJysrKinJPAAAAAACJ5ODBg0pLS+vxvM3oK9YngPb2dn322WcaOXKkbDZbtLsTkNvtVlZWllwul1JTU6PdHYQJ1zn+cY3jH9c4/nGN4x/XODFwneOf1a+xYRg6ePCgMjMzZbf3vHKdkXZJdrtdJ554YrS7EZTU1FRL/sAhtLjO8Y9rHP+4xvGPaxz/uMaJgesc/6x8jXsbYe9AIToAAAAAACyK0A4AAAAAgEUR2mNESkqK7rjjDqWkpES7KwgjrnP84xrHP65x/OMaxz+ucWLgOse/eLnGFKIDAAAAAMCiGGkHAAAAAMCiCO0AAAAAAFgUoR0AAAAAAIsitAMAAAAAYFGEdotZt26dJk6cqNTUVKWmpuq8887Ts88+6ztvGIaWL1+uzMxMOZ1OFRQUaNeuXVHsMQZj5cqVstlsWrhwoe8Y1zj2LV++XDabze+RkZHhO881jg+ffvqprr76ah177LEaNmyYzjzzTL3++uu+81zn2DZ69Ohu/45tNpsWLFggiesbD1pbW/XrX/9aY8aMkdPpVHZ2tkpKStTe3u5rw3WODwcPHtTChQt10kknyel0Ki8vTzt27PCd5zrHlpdeekmXXnqpMjMzZbPZ9Oc//9nvfDDXs6WlRb/4xS903HHHafjw4brsssv0ySefRPC76B9Cu8WceOKJuvvuu/Xaa6/ptdde0/nnn6+ZM2f6ftBWrVql0tJSrV27Vjt27FBGRoZmzJihgwcPRrnn6K8dO3boD3/4gyZOnOh3nGscH04//XTt3bvX96ivr/ed4xrHvgMHDig/P19DhgzRs88+q3fffVf33XefjjrqKF8brnNs27Fjh9+/4a1bt0qSfvzjH0vi+saDe+65R+vXr9fatWu1e/durVq1Svfee68efPBBXxuuc3y44YYbtHXrVpWXl6u+vl4XXHCBCgsL9emnn0riOsear776SmeccYbWrl0b8Hww13PhwoWqrKzU008/rZqaGh06dEiXXHKJ2traIvVt9I8Byzv66KONhx9+2GhvbzcyMjKMu+++23fu66+/NtLS0oz169dHsYfor4MHDxpjx441tm7dakyfPt246aabDMMwuMZx4o477jDOOOOMgOe4xvHh1ltvNaZMmdLjea5z/LnpppuMk08+2Whvb+f6xomLL77YmDt3rt+xoqIi4+qrrzYMg3/H8eLw4cNGUlKS8cwzz/gdP+OMM4zbb7+d6xzjJBmVlZW+18Fczy+//NIYMmSI8fTTT/vafPrpp4bdbjeee+65iPW9Pxhpt7C2tjY9/fTT+uqrr3TeeeepsbFRTU1NuuCCC3xtUlJSNH36dNXV1UWxp+ivBQsW6OKLL1ZhYaHfca5x/Hj//feVmZmpMWPG6IorrtCePXskcY3jxV/+8hdNnjxZP/7xj3XCCSdo0qRJeuihh3znuc7xxev16sknn9TcuXNls9m4vnFiypQpeuGFF9TQ0CBJeuutt1RTU6Mf/OAHkvh3HC9aW1vV1tamoUOH+h13Op2qqanhOseZYK7n66+/rm+++cavTWZmpiZMmGDZa05ot6D6+nqNGDFCKSkpmjdvniorKzV+/Hg1NTVJktLT0/3ap6en+87B+p5++mm98cYbWrlyZbdzXOP4cM455+iJJ57Q888/r4ceekhNTU3Ky8vTF198wTWOE3v27NG6des0duxYPf/885o3b55uvPFGPfHEE5L4txxv/vznP+vLL7/UnDlzJHF948Wtt96qK6+8UqeeeqqGDBmiSZMmaeHChbryyislcZ3jxciRI3Xeeefpzjvv1Geffaa2tjY9+eSTeuWVV7R3716uc5wJ5no2NTUpOTlZRx99dI9trMYR7Q6gu3Hjxmnnzp368ssv9ac//UnXXnuttm/f7jtvs9n82huG0e0YrMnlcummm27Sli1but3x7YxrHNsuuugi359zc3N13nnn6eSTT9aGDRt07rnnSuIax7r29nZNnjxZK1askCRNmjRJu3bt0rp163TNNdf42nGd48Mjjzyiiy66SJmZmX7Hub6xraKiQk8++aSeeuopnX766dq5c6cWLlyozMxMXXvttb52XOfYV15errlz5+pb3/qWkpKSdNZZZ+mqq67SG2+84WvDdY4vA7meVr7mjLRbUHJysk455RRNnjxZK1eu1BlnnKH777/fV3266x2g/fv3d7ubBGt6/fXXtX//fp199tlyOBxyOBzavn27HnjgATkcDt915BrHl+HDhys3N1fvv/8+/47jxKhRozR+/Hi/Y6eddpo+/vhjSeI6x5GPPvpI27Zt0w033OA7xvWND0uWLNFtt92mK664Qrm5uSouLtaiRYt8M+G4zvHj5JNP1vbt23Xo0CG5XC69+uqr+uabbzRmzBiuc5wJ5npmZGTI6/XqwIEDPbaxGkJ7DDAMQy0tLb7/WDoq2ErmOrvt27crLy8vij1EsL73ve+pvr5eO3fu9D0mT56sn/zkJ9q5c6eys7O5xnGopaVFu3fv1qhRo/h3HCfy8/P13nvv+R1raGjQSSedJElc5zjy2GOP6YQTTtDFF1/sO8b1jQ+HDx+W3e7/VjgpKcm35RvXOf4MHz5co0aN0oEDB/T8889r5syZXOc4E8z1PPvsszVkyBC/Nnv37tU777xj3WserQp4CGzp0qXGSy+9ZDQ2Nhpvv/228atf/cqw2+3Gli1bDMMwjLvvvttIS0sz/ud//seor683rrzySmPUqFGG2+2Ocs8xUJ2rxxsG1zge3HzzzUZVVZWxZ88e4+WXXzYuueQSY+TIkcaHH35oGAbXOB68+uqrhsPhMO666y7j/fffN/7rv/7LGDZsmPHkk0/62nCdY19bW5vx7W9/27j11lu7neP6xr5rr73W+Na3vmU888wzRmNjo/E///M/xnHHHWfccsstvjZc5/jw3HPPGc8++6yxZ88eY8uWLcYZZ5xhfPe73zW8Xq9hGFznWHPw4EHjzTffNN58801DklFaWmq8+eabxkcffWQYRnDXc968ecaJJ55obNu2zXjjjTeM888/3zjjjDOM1tbWaH1bvSK0W8zcuXONk046yUhOTjaOP/5443vf+54vsBuGuY3BHXfcYWRkZBgpKSnGtGnTjPr6+ij2GIPVNbRzjWPf7NmzjVGjRhlDhgwxMjMzjaKiImPXrl2+81zj+LB582ZjwoQJRkpKinHqqacaf/jDH/zOc51j3/PPP29IMt57771u57i+sc/tdhs33XST8e1vf9sYOnSokZ2dbdx+++1GS0uLrw3XOT5UVFQY2dnZRnJyspGRkWEsWLDA+PLLL33nuc6x5a9//ashqdvj2muvNQwjuOvp8XiMn//858YxxxxjOJ1O45JLLjE+/vjjKHw3wbEZhmFEcaAfAAAAAAD0gDXtAAAAAABYFKEdAAAAAACLIrQDAAAAAGBRhHYAAAAAACyK0A4AAAAAgEUR2gEAAAAAsChCOwAAAAAAFkVoBwAAAADAogjtAABYwJw5c2Sz2bo9Lrzwwmh3LaCbbrpJZ599tlJSUnTmmWcG9TFvvvmmLrnkEp1wwgkaOnSoRo8erdmzZ+vzzz8Pb2cBAIhhjmh3AAAAmC688EI99thjfsdSUlKi1JveGYahuXPn6pVXXtHbb7/dZ/v9+/ersLBQl156qZ5//nkdddRRamxs1F/+8hcdPnw4bP385ptvNGTIkLB9fgAAwo2RdgAALCIlJUUZGRl+j6OPPlqSVFVVpeTkZFVXV/va33fffTruuOO0d+9eSdJzzz2nKVOm6KijjtKxxx6rSy65RB988IGv/YcffiibzaY//vGPmjp1qpxOp77zne+ooaFBO3bs0OTJkzVixAhdeOGF+sc//tFrXx944AEtWLBA2dnZQX1vdXV1crvdevjhhzVp0iSNGTNG559/vtasWaNvf/vbvna7du3SxRdfrNTUVI0cOVJTp071fQ/t7e0qKSnRiSee6Bvhf+655wJ+fwUFBRo6dKiefPJJSdJjjz2m0047TUOHDtWpp56qsrKyoPoNAEC0EdoBAIgBBQUFWrhwoYqLi9Xc3Ky33npLt99+ux566CGNGjVKkvTVV19p8eLF2rFjh1544QXZ7XbNmjVL7e3tfp/rjjvu0K9//Wu98cYbcjgcuvLKK3XLLbfo/vvvV3V1tT744AMtW7YspP3PyMhQa2urKisrZRhGwDaffvqppk2bpqFDh+rFF1/U66+/rrlz56q1tVWSdP/99+u+++7T73//e7399tv6/ve/r8suu0zvv/++3+e59dZbdeONN2r37t36/ve/r4ceeki333677rrrLu3evVsrVqzQb37zG23YsCGk3yMAAGFhAACAqLv22muNpKQkY/jw4X6PkpISX5uWlhZj0qRJxuWXX26cfvrpxg033NDr59y/f78hyaivrzcMwzAaGxsNScbDDz/sa/Pf//3fhiTjhRde8B1buXKlMW7cuKD6fccddxhnnHFGUG1/9atfGQ6HwzjmmGOMCy+80Fi1apXR1NTkO7906VJjzJgxhtfrDfjxmZmZxl133eV37Dvf+Y4xf/58v+9vzZo1fm2ysrKMp556yu/YnXfeaZx33nlB9RsAgGhipB0AAIv4t3/7N+3cudPvsWDBAt/55ORkPfnkk/rTn/4kj8ejNWvW+H38Bx98oKuuukrZ2dlKTU3VmDFjJEkff/yxX7uJEyf6/pyeni5Jys3N9Tu2f//+UH97uuuuu9TU1KT169dr/PjxWr9+vU499VTV19dLknbu3KmpU6cGXIPudrv12WefKT8/3+94fn6+du/e7Xds8uTJvj//4x//kMvl0vXXX68RI0b4Hr/73e/8lg4AAGBVFKIDAMAihg8frlNOOaXXNnV1dZKkf/7zn/rnP/+p4cOH+85deumlysrK0kMPPaTMzEy1t7drwoQJ8nq9fp+jcyi22WwBj3WdUh8qxx57rH784x/rxz/+sVauXKlJkybp97//vTZs2CCn09nnx3f0t4NhGN2Odf476fg+HnroIZ1zzjl+7ZKSkgb6bQAAEDGMtAMAECM++OADLVq0SA899JDOPfdcXXPNNb5Q+sUXX2j37t369a9/re9973s67bTTdODAgSj3uHfJyck6+eST9dVXX0kyZwBUV1frm2++6dY2NTVVmZmZqqmp8TteV1en0047rcevkZ6erm9961vas2ePTjnlFL9Hx0wEAACsjJF2AAAsoqWlRU1NTX7HHA6HjjvuOLW1tam4uFgXXHCBrrvuOl100UXKzc3VfffdpyVLlujoo4/Wscceqz/84Q8aNWqUPv74Y912221h6+vf//53HTp0SE1NTfJ4PNq5c6ckafz48UpOTu7W/plnntHTTz+tK664Qjk5OTIMQ5s3b9b/+3//z7fN3c9//nM9+OCDuuKKK7R06VKlpaXp5Zdf1ne/+12NGzdOS5Ys0R133KGTTz5ZZ555ph577DHt3LlT//Vf/9VrX5cvX64bb7xRqampuuiii9TS0qLXXntNBw4c0OLFi0P+dwMAQCgR2gEAsIjnnnvOVwm+w7hx4/T//X//n+666y59+OGH2rx5sySzGvvDDz+syy+/XDNmzNCZZ56pp59+WjfeeKMmTJigcePG6YEHHlBBQUFY+nrDDTdo+/btvteTJk2SJDU2Nmr06NHd2o8fP17Dhg3TzTffLJfLpZSUFI0dO1YPP/ywiouLJZlT51988UUtWbJE06dPV1JSks4880zfOvYbb7xRbrdbN998s/bv36/x48frL3/5i8aOHdtnX4cNG6Z7771Xt9xyi4YPH67c3FwtXLgwNH8ZAACEkc0weth3BQAAAAAARBVr2gEAAAAAsChCOwAAAAAAFkVoBwAAAADAogjtAAAAAABYFKEdAAAAAACLIrQDAAAAAGBRhHYAAAAAACyK0A4AAAAAgEUR2gEAAAAAsChCOwAAAAAAFkVoBwAAAADAogjtAAAAAABY1P8PBAv/3+/8i8AAAAAASUVORK5CYII="
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "execution_count": 3
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "看起来在两类间，有一个清晰的决策边界。现在我们需要实现逻辑回归，那样就可以训练一个模型来预测结果。方程实现在下面的代码示例在\"exercises\" 文件夹的 \"ex2.pdf\" 中。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# sigmoid 函数\n",
    "g 代表一个常用的逻辑函数（logistic function）为S形函数（Sigmoid function），公式为： \\\\[g\\left( z \\right)=\\frac{1}{1+{{e}^{-z}}}\\\\] \n",
    "合起来，我们得到逻辑回归模型的假设函数： \n",
    "\t\\\\[{{h}_{\\theta }}\\left( x \\right)=\\frac{1}{1+{{e}^{-{{\\theta }^{T}}X}}}\\\\] "
   ]
  },
  {
   "cell_type": "code",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2024-08-07T02:04:38.065152Z",
     "start_time": "2024-08-07T02:04:38.062588Z"
    }
   },
   "source": [
    "def sigmoid(z):\n",
    "    return 1 / (1 + np.exp(-z))"
   ],
   "outputs": [],
   "execution_count": 4
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "让我们做一个快速的检查，来确保它可以工作。"
   ]
  },
  {
   "cell_type": "code",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2024-08-07T02:04:38.205557Z",
     "start_time": "2024-08-07T02:04:38.085920Z"
    }
   },
   "source": [
    "nums = np.arange(-10, 10, step=1)\n",
    "\n",
    "fig, ax = plt.subplots(figsize=(12,8))\n",
    "ax.plot(nums, sigmoid(nums), 'r')\n",
    "plt.show()"
   ],
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<Figure size 1200x800 with 1 Axes>"
      ],
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAA9UAAAKTCAYAAAAXPCvaAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjguNCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8fJSN1AAAACXBIWXMAAA9hAAAPYQGoP6dpAABQvElEQVR4nO3de7hVdYE+8PdwFxVUUIREJDVzxEohFdRJbcTwnoZoM2o3Z6y0FGcaUSfzUlTjWFOTZr9QKx3De5KmYnnHShksSyc1SRRBwgugKNf1+2PNAZGLnMM5Z+19zufzPPvZ3732Wvu8u93inNf13Ws1FEVRBAAAAGiyTlUHAAAAgHqlVAMAAEAzKdUAAADQTEo1AAAANJNSDQAAAM2kVAMAAEAzKdUAAADQTF2qDrA+li9fnhdeeCGbbrppGhoaqo4DAABAO1cURRYsWJABAwakU6e1H4+ui1L9wgsvZODAgVXHAAAAoIN57rnnss0226z1+boo1ZtuummS8s306tWr4jQAAAC0d/Pnz8/AgQNX9NG1qYtS3Tjlu1evXko1AAAAbeadvoLsRGUAAADQTEo1AAAANJNSDQAAAM2kVAMAAEAzKdUAAADQTEo1AAAANJNSDQAAAM2kVAMAAEAzKdUAAADQTEo1AAAANJNSDQAAAM2kVAMAAEAzKdUAAADQTEo1AAAANJNSDQAAAM2kVAMAAEAzKdUAAADQTEo1AAAANJNSDQAAAM2kVAMAAEAzKdUAAADQTE0u1ffdd18OO+ywDBgwIA0NDbn55pvfcZt77703Q4cOTY8ePfLud7873//+95uTFQAAAGpKk0v166+/nve///35r//6r/Vaf/r06Tn44IOz7777Ztq0aTnrrLPyhS98ITfccEOTwwIAAEAt6dLUDUaNGpVRo0at9/rf//73s+222+bb3/52kmTnnXfOI488kosuuihHH310U388AAAA1Iwml+qmeuihhzJy5MhVlh100EGZMGFClixZkq5du662zaJFi7Jo0aIVj+fPn9/aMQEAgPZg2bJkyZL1vy1evOrjpUuT5cvXfCuKllneGq9VFOX7bxy//XFLjVvqtQ45JBk3ruU+9wq1eqmePXt2+vXrt8qyfv36ZenSpZk7d2769++/2jbjx4/Peeed19rRAACADbF8efL668n8+cmCBeX9W8cLFiQLF65/oW2JW2Nxo7bttFPVCVpMq5fqJGloaFjlcfF//0d/+/JG48aNy9ixY1c8nj9/fgYOHNh6AQEAoKMoirLorq0EN2XZa6/VR4nt0iXp2nXdt27dyvsuXZLOnZNOnVa9NTSsvmx9n2/uc+u7bUPDyluy9sctNW6J19puuxb7eKvW6qV66623zuzZs1dZNmfOnHTp0iV9+vRZ4zbdu3dP9+7dWzsaAADUj6JIXn45eeWV5pfgxvHy5S2brXPnpFevZNNNy/u3jnv2fOdC25q3Ll1WFjloBa1eqocPH55JkyatsuzOO+/MsGHD1vh9agAA6LCKInnxxeTpp8vbU0+ter9gQcv9rIaGlcV3TWV4TcvW9nyPHoorHVaTS/Vrr72Wp59+esXj6dOn59FHH80WW2yRbbfdNuPGjcvMmTPz4x//OEly8skn57/+678yduzYnHTSSXnooYcyYcKEXHPNNS33LgAAoF4URTJ79qpl+a3j115b9/Ybb7z+ZXddy3r2LKcOAxukyaX6kUceyf7777/iceN3n0888cRceeWVmTVrVmbMmLHi+cGDB+e2227L6aefnu9973sZMGBAvvOd77icFgAA7VdRJLNmrbk0P/10eXKvtWloSLbdNtlxx2SHHVbe77BD8u53l0eFgZrRUBS1f2aB+fPnp3fv3pk3b1569epVdRwAACi/l/zW4vzW0vz00+XJwNamU6dk0KDVS/OOOyaDByfOLwSVW98e2iZn/wYAgLq0fHkyc+aav+P85z8nb7yx9m07dSrPcPz20rzDDuVyxRnaBaUaAICObfny5Pnn1/wd5z//OXnzzbVv27lzeWT57aW5sTh369ZmbwOohlINAEDHsnx5cv/9yU9+kjz0UFmcFy1a+/pduqwszm8/6jxoUHnZJqDDUqoBAOgYnn66LNI//nHyl7+s+lyXLuVJwNY0VXvQoPJ5gDXwrwMAAO3XvHnJtdcmP/pR8uCDK5dvumlyzDHJUUcl731vebZtxRloBv9yAADQvixblkyeXBbpm29e+Z3oTp2SAw9MTjghOfLI8jrNABtIqQYAoH34wx/Kqd1XXVVe6qrR3/xNcuKJyd//ffKud1WXD2iXlGoAAOrXX/+aXHNNeVT6f/5n5fI+fZLjjivL9NChSUNDdRmBdk2pBgCgvixenNx6a1mkb701Wbq0XN6lS3LooeX07kMOcTkroE0o1QAA1L6iSB55pCzS11yTvPzyyueGDi2PSB93XNK3b3UZgQ5JqQYAoHbNnFl+R/pHP0qeeGLl8v79k3/4h7JM77JLdfmADk+pBgCgtixcmNx0U3nSsbvuSpYvL5f36JF89KNlkf7wh10CC6gJ/iUCAKB6RZHcf395RPq665IFC1Y+t88+ZZEePTrp3bu6jABroFQDAFCdZ54pj0j/+MfJ9Okrl2+3XXnCsRNOSLbfvrJ4AO9EqQYAoG3Nm5dcf315VPr++1cu33TT8mj0CSck++6bdOpUXUaA9aRUAwDQ+pYtK78f/aMfld+XfvPNcnlDQ/J3f1dO7/7oR5OePavNCdBESjUAAK3n8cfLIn3VVckLL6xc/t73lkX6H/4h2Wab6vIBbCClGgCAljV3bnkt6R/9KJk6deXyLbYoryV94onJsGHlUWqAOqdUAwCw4YoimTQpueKK5NZbkyVLyuVduiQHH1wW6UMOSbp3rzYnQAtTqgEA2DCLFyef/GTy3/+9ctluu5VF+rjjkq22qi4bQCtTqgEAaL4FC5Kjj04mTy6PSn/hC8knPpHsumvVyQDahFINAEDzzJlTTu2eOjXZeOPkhhuSgw6qOhVAm1KqAQBoumeeKQv0008nffsmt92WfPCDVacCaHNKNQAATTNtWjJqVPLii8l22yV33JG85z1VpwKoRKeqAwAAUEd+9avkQx8qC/X7359MmaJQAx2aUg0AwPq59trkIx8pT062337Jvfcm/ftXnQqgUko1AADv7LvfTY49trz+9Mc+ltx+e9K7d9WpACqnVAMAsHZFkZx9dnmprKJIPv/55Kc/Tbp3rzoZQE1wojIAANZs6dLkn/4pufzy8vGFFyZnnZU0NFSbC6CGKNUAAKxu4cJkzJjk5z9POnVKLrss+cxnqk4FUHOUagAAVvXSS8lhhyUPPZT06JFMnJgcfnjVqQBqklINAMBKzz2XHHRQ8sQTyeabJ5MmJXvvXXUqgJqlVAMAUPrjH8tCPXNmss025Rm+d9ml6lQANc3ZvwEASB54INlnn7JQ77xzMmWKQg2wHpRqAICO7pZbkgMPTF59NRkxoizYAwdWnQqgLijVAAAd2Q9/mHz0o8mbbyaHHppMnpxssUXVqQDqhlINANARFUVywQXJSScly5cnn/pUctNNSc+eVScDqCtKNQBAR7NsWXLKKcmXv1w+Pvvs8oh1F+ewBWgq/3ICAHQkb76ZHH98cv31SUND8p3vlAUbgGZRqgEAOop585IjjkjuvTfp1i35yU+SY46pOhVAXVOqAQA6ghdeSEaNSn7/+2TTTZOf/SzZf/+qUwHUPaUaAKC9e/LJZOTI5Nlnk379kl/8Itltt6pTAbQLTlQGANCe/fa3yd57l4V6hx2SKVMUaoAWpFQDALRXt99eTvGeOzcZNix58MHk3e+uOhVAu6JUAwC0Rz/5SXLYYcnChcmBBya/+lWy1VZVpwJod5RqAID25qKLkhNOSJYuTT7+8eTnPy9PTgZAi1OqAQDai+XLkzPOSP7lX8rHY8eWR6y7das2F0A75uzfAADtweLFyac+lVx9dfn43/89+ed/rjYTQAegVAMA1LvXXkuOPjq5886kS5fk8suT44+vOhVAh6BUAwDUszlzkkMOSR55JOnZM7nhhuQjH6k6FUCHoVQDANSr6dOTkSOTp59O+vRJbr012XPPqlMBdChKNQBAPXr00WTUqGT27GTQoOSOO5Kddqo6FUCH4+zfAAD15u67k7/927JQv+99yZQpCjVARZRqAIB6ct115XemFywoi/W99yYDBlSdCqDDUqoBAOrF976XjBlTXj7rqKPKKd+bbVZ1KoAOTakGAKh1RZGcc05yyinl+LOfTa69NunRo+pkAB2eE5UBANSypUuTk09OJkwoH59/flmwGxqqzQVAEqUaAKB2LVyYHHdccsstSadOyaWXJv/4j1WnAuAtlGoAgFr08svJYYeVZ/bu0SO55prkyCOrTgXA2yjVAAC1Ztmy5KCDkkceKU9Edsstyb77Vp0KgDVQqgEAas1VV60s1PffnwwZUnUiANbC2b8BAGrJokXJueeW43HjFGqAGqdUAwDUkssuS559NhkwoLyEFgA1TakGAKgVCxYkF15Yjs89N+nZs9o8ALwjpRoAoFZ861vJX/+a7Lhj8slPVp0GgPWgVAMA1IK//jW56KJyfOGFSdeu1eYBYL0o1QAAtWD8+HL69267JR/7WNVpAFhPSjUAQNVmzEguuaQcjx+fdPInGkC98C82AEDVzjuvvJTWfvslI0dWnQaAJlCqAQCq9MQTyZVXluPx45OGhkrjANA0SjUAQJXOOSdZvjw58shkr72qTgNAEynVAABV+e1vkxtvLL9D3Xh9agDqilINAFCVcePK++OPT3bZpdosADSLUg0AUIW77kp+9aukW7fkK1+pOg0AzaRUAwC0taJIzjyzHH/2s8l221UaB4DmU6oBANraDTckU6cmm2ySnHVW1WkA2ABKNQBAW1q6NDn77HJ8xhnJVltVmweADaJUAwC0pSuvTJ58MunbNxk7tuo0AGwgpRoAoK288cbKk5KddVbSq1elcQDYcEo1AEBbueSSZObMZODA8gRlANQ9pRoAoC3Mm5d87Wvl+Lzzkh49qs0DQItQqgEA2sJFFyUvv5zsvHNy/PFVpwGghSjVAACt7cUXk4svLsdf/WrSpUu1eQBoMUo1AEBru/DCZOHCZI89kiOPrDoNAC1IqQYAaE3TpyeXXVaOx49PGhqqzQNAi1KqAQBa07nnJkuWJAcemBxwQNVpAGhhSjUAQGt57LHkqqvKceOZvwFoV5RqAIDWcvbZSVEko0cnw4ZVnQaAVqBUAwC0hgcfTCZNSjp3Ti64oOo0ALQSpRoAoKUVRXLmmeX4U59Kdtqp2jwAtBqlGgCgpf3iF8kDDyTduydf/nLVaQBoRUo1AEBLWr48Oeuscnzqqck221SbB4BWpVQDALSkiROT3/0u6dVr5RRwANotpRoAoKUsXpycc045/tKXkj59qs0DQKtTqgEAWsqECckzzyT9+iVf/GLVaQBoA0o1AEBLeP315Pzzy/G//VuyySbV5gGgTSjVAAAt4TvfSWbPTgYPTk46qeo0ALQRpRoAYEO98kryzW+W4/PPT7p1qzYPAG1GqQYA2FDf+Eby6qvJrrsmxx1XdRoA2pBSDQCwIWbOTP7zP8vx176WdO5cbR4A2pRSDQCwIS64IHnzzWTvvZNDDqk6DQBtrFml+pJLLsngwYPTo0ePDB06NPfff/8617/66qvz/ve/Pz179kz//v3zyU9+Mi+99FKzAgMA1Iynnkp++MNy/PWvJw0N1eYBoM01uVRPnDgxp512Ws4+++xMmzYt++67b0aNGpUZM2ascf0HHnggJ5xwQj796U/nj3/8Y6677ro8/PDD+cxnPrPB4QEAKvVv/5YsW5YcfHCyzz5VpwGgAg1FURRN2WDPPffM7rvvnksvvXTFsp133jlHHnlkxo8fv9r6F110US699NL8+c9/XrHsu9/9br75zW/mueeeW6+fOX/+/PTu3Tvz5s1Lr169mhIXAKB1TJuW7L57OX700eT97680DgAta317aJOOVC9evDhTp07NyJEjV1k+cuTITJkyZY3bjBgxIs8//3xuu+22FEWRF198Mddff30OWcd3jhYtWpT58+evcgMAqClnnVXef/zjCjVAB9akUj137twsW7Ys/fr1W2V5v379Mnv27DVuM2LEiFx99dUZM2ZMunXrlq233jqbbbZZvvvd767154wfPz69e/decRs4cGBTYgIAtK577kluvz3p0qW8LjUAHVazTlTW8LaTcBRFsdqyRo8//ni+8IUv5Mtf/nKmTp2a22+/PdOnT8/JJ5+81tcfN25c5s2bt+K2vtPEAQBaXVEk48aV43/8x2T77avNA0ClujRl5b59+6Zz586rHZWeM2fOakevG40fPz577713/uVf/iVJ8r73vS8bb7xx9t1331x44YXp37//att079493bt3b0o0AIC2ccstya9/nfTsmZxzTtVpAKhYk45Ud+vWLUOHDs3kyZNXWT558uSMGDFijdssXLgwnTqt+mM6d+6cpDzCDQBQN5YtW/ld6i9+MVnDwQEAOpYmT/8eO3ZsfvjDH+byyy/PE088kdNPPz0zZsxYMZ173LhxOeGEE1asf9hhh+XGG2/MpZdemmeeeSYPPvhgvvCFL2SPPfbIgAEDWu6dAAC0tquvTh5/PNl88+RLX6o6DQA1oEnTv5NkzJgxeemll3L++edn1qxZGTJkSG677bYMGjQoSTJr1qxVrln9iU98IgsWLMh//dd/5Ywzzshmm22WAw44IN/4xjda7l0AALS2RYuSL3+5HJ95ZrLZZpXGAaA2NPk61VVwnWoAoHLf+U455XvAgOSpp8rvVAPQbrXKdaoBADqkBQuSCy8sx+eeq1ADsIJSDQDwTr71reSvf0123DH55CerTgNADVGqAQDWZe7c5KKLyvEFFyRdu1abB4CaolQDAKzL+PHl9O/ddktGj646DQA1RqkGAFibGTOS732vHI8fn3TypxMAq/KbAQBgbc47r7yU1n77JSNHVp0GgBqkVAMArMkTTyRXXlmOx49PGhoqjQNAbVKqAQDW5JxzkuXLkyOPTPbaq+o0ANQopRoA4O1++9vkxhvLo9ON16cGgDVQqgEA3u6ss8r7E05Idtml2iwA1DSlGgDgre66K/nlL5Nu3ZKvfKXqNADUOKUaAKBRUSRnnlmOP/vZZLvtKo0DQO1TqgEAGt1wQzJ1arLJJiungAPAOijVAABJsnRpcvbZ5fiMM5Kttqo2DwB1QakGAEjKa1I/+WTSp08ydmzVaQCoE0o1AMAbbyTnnVeOzz476dWr2jwA1A2lGgDgkkuS559PBg4sT1AGAOtJqQYAOrZ585Kvfa0cn3de0qNHtXkAqCtKNQDQsV10UfLyy8nOOyfHH191GgDqjFINAHRcL76YXHxxOb7wwqRLl2rzAFB3lGoAoOO68MJk4cLkgx9MPvrRqtMAUIeUagCgY5o+PbnssnL89a8nDQ3V5gGgLinVAEDHdO65yZIlyYEHJgccUHUaAOqUUg0AdDyPPZZcdVU5bjzzNwA0g1INAHQ8Z5+dFEUyenQybFjVaQCoY0o1ANCxPPhgMmlS0rlzcsEFVacBoM4p1QBAx1EUybhx5fiTn0x22qnaPADUPaUaAOg4br89uf/+pHv38kRlALCBlGoAoGNYvnzlUepTT0222abaPAC0C0o1ANAxTJyY/O53Sa9eyZlnVp0GgHZCqQYA2r/Fi5NzzinHX/pS0qdPtXkAaDeUagCg/fvv/06eeSbZaqvki1+sOg0A7YhSDQC0f1ddVd5/8YvJJptUmwWAdkWpBgDat9mzk7vvLscf/3i1WQBod5RqAKB9u/768szfe+2VbLdd1WkAaGeUagCgffvpT8v7Y4+tNgcA7ZJSDQC0XzNmJA8+mDQ0JKNHV50GgHZIqQYA2q9rry3vP/ShZMCAarMA0C4p1QBA+2XqNwCtTKkGANqnp55Kpk5NOndOjj666jQAtFNKNQDQPk2cWN4feGDSt2+1WQBot5RqAKB9MvUbgDagVAMA7c8f/pD88Y9Jt27JkUdWnQaAdkypBgDan2uuKe8PPjjp3bvaLAC0a0o1ANC+FIWp3wC0GaUaAGhfHnkkeeaZpGfP5NBDq04DQDunVAMA7UvjUerDD0823rjaLAC0e0o1ANB+LF++8lJapn4D0AaUagCg/XjwwWTmzPLkZB/5SNVpAOgAlGoAoP1onPr90Y8m3btXmwWADkGpBgDah6VLk+uuK8emfgPQRpRqAKB9uPvu5K9/Tfr2TQ44oOo0AHQQSjUA0D40Tv3+2MeSrl2rzQJAh6FUAwD1b9Gi5MYby7Gp3wC0IaUaAKh/d96ZvPpqMmBAss8+VacBoANRqgGA+tc49fuYY5LOnavNAkCHolQDAPVt4cLkZz8rx6Z+A9DGlGoAoL7demvy+uvJdtsle+xRdRoAOhilGgCob41Tv489NmloqDYLAB2OUg0A1K/588sj1Ymp3wBUQqkGAOrXz35WXk7rve9N3ve+qtMA0AEp1QBA/TL1G4CKKdUAQH166aXy+tRJMmZMtVkA6LCUagCgPt14Y7J0afKBD5TTvwGgAko1AFCf3jr1GwAqolQDAPVn1qzk7rvLsanfAFRIqQYA6s/11ydFkey1V7LddlWnAaADU6oBgPpj6jcANUKpBgDqy7PPJlOmlJfQGj266jQAdHBKNQBQX669trzfb79kwIBKowCAUg0A1BdTvwGoIUo1AFA/nnwy+Z//Sbp0SY46quo0AKBUAwB1ZOLE8v7AA5O+favNAgBRqgGAelEUyTXXlGNTvwGoEUo1AFAf/vCH5Iknku7dkyOOqDoNACRRqgGAetF4grKDD0569642CwD8H6UaAKh9ReGs3wDUJKUaAKh9jzySPPNMsvHGySGHVJ0GAFZQqgGA2td4lPrww8tiDQA1QqkGAGrb8uUrL6Vl6jcANUapBgBq2wMPJDNnlicnO+igqtMAwCqUagCgtjVO/T7qqPJyWgBQQ5RqAKB2LV2aXHddOTb1G4AapFQDALXrV79K5s5N+vZNDjig6jQAsBqlGgCoXY1Tv0ePTrp0qTYLAKyBUg0A1KZFi5IbbyzHpn4DUKOUagCgNt1xRzJvXjJgQLLPPlWnAYA1UqoBgNrUOPV7zJikkz9ZAKhNfkMBALXn9deTn/2sHJv6DUANU6oBgNpz663JwoXJ4MHJBz9YdRoAWCulGgCoPY1Tv489NmloqDYLAKyDUg0A1JZ585LbbivHpn4DUOOUagCgtvzsZ+XltHbeOdl116rTAMA6KdUAQG0x9RuAOqJUAwC1Y+7cZPLkcjxmTLVZAGA9KNUAQO248cZk6dJkt92SnXaqOg0AvCOlGgCoHW+d+g0AdUCpBgBqw6xZyT33lONjjqk0CgCsL6UaAKgN112XFEUyfHiy3XZVpwGA9aJUAwC1wdRvAOqQUg0AVO8vf0keeqi8hNbo0VWnAYD11qxSfckll2Tw4MHp0aNHhg4dmvvvv3+d6y9atChnn312Bg0alO7du2f77bfP5Zdf3qzAAEA7dO215f1++yX9+1caBQCaoktTN5g4cWJOO+20XHLJJdl7771z2WWXZdSoUXn88cez7bbbrnGbY445Ji+++GImTJiQHXbYIXPmzMnSpUs3ODwA0E6Y+g1AnWooiqJoygZ77rlndt9991x66aUrlu2888458sgjM378+NXWv/3223PsscfmmWeeyRZbbNGskPPnz0/v3r0zb9689OrVq1mvAQDUqD/9KXnve5MuXcozgPftW3UiAFjvHtqk6d+LFy/O1KlTM3LkyFWWjxw5MlOmTFnjNrfcckuGDRuWb37zm3nXu96V97znPfnnf/7nvPHGG2v9OYsWLcr8+fNXuQEA7dTEieX9gQcq1ADUnSZN/547d26WLVuWfv36rbK8X79+mT179hq3eeaZZ/LAAw+kR48euemmmzJ37tx87nOfy8svv7zW71WPHz8+5513XlOiAQD1qCiSa64px6Z+A1CHmnWisoaGhlUeF0Wx2rJGy5cvT0NDQ66++ursscceOfjgg3PxxRfnyiuvXOvR6nHjxmXevHkrbs8991xzYgIAte6xx5L//d+ke/fkiCOqTgMATdakI9V9+/ZN586dVzsqPWfOnNWOXjfq379/3vWud6V3794rlu28884piiLPP/98dtxxx9W26d69e7p3796UaABAPWo8QdnBBydv+VsBAOpFk45Ud+vWLUOHDs3kyZNXWT558uSMGDFijdvsvffeeeGFF/Laa6+tWPbkk0+mU6dO2WabbZoRGQBoF4rCWb8BqHtNnv49duzY/PCHP8zll1+eJ554IqeffnpmzJiRk08+OUk5dfuEE05Ysf7HP/7x9OnTJ5/85Cfz+OOP57777su//Mu/5FOf+lQ22mijlnsnAEB9efjhZPr0ZOONk0MOqToNADRLk69TPWbMmLz00ks5//zzM2vWrAwZMiS33XZbBg0alCSZNWtWZsyYsWL9TTbZJJMnT86pp56aYcOGpU+fPjnmmGNy4YUXtty7AADqT+NR6sMPL4s1ANShJl+nugquUw0A7czy5cnAgckLLyQ/+1lZrAGghrTKdaoBAFrEAw+Uhbp37+Sgg6pOAwDNplQDAG2vcer3UUeVl9MCgDqlVAMAbWvp0uS668qxs34DUOeUagCgbf3qV8ncuUnfvskBB1SdBgA2iFINALSta64p70ePTro0+UIkAFBTlGoAoO0sWpTceGM5NvUbgHZAqQYA2s7ttyfz5ycDBiT77FN1GgDYYEo1ANB2Gs/6PWZM0smfIQDUP7/NAIC28frryS23lGNTvwFoJ5RqAKBt/PznycKFybvfnXzwg1WnAYAWoVQDAG2jcer3sccmDQ3VZgGAFqJUAwCtb9685LbbyrGp3wC0I0o1AND6br45Wbw4+Zu/SYYMqToNALQYpRoAaH2mfgPQTinVAEDrmjs3mTy5HI8ZU20WAGhhSjUA0LpuuCFZtizZfffkPe+pOg0AtCilGgBoXW+d+g0A7YxSDQC0nhdeSO69txwfc0y1WQCgFSjVAEDrue66pCiSESOSQYOqTgMALU6pBgBaj6nfALRzSjUA0DqmT09+/eukU6dk9Oiq0wBAq1CqAYDWce215f1++yVbb11pFABoLUo1ANA6TP0GoANQqgGAlve//5s8+mjSpUty1FFVpwGAVqNUAwAtb+LE8n7kyKRPn2qzAEArUqoBgJZVFKZ+A9BhKNUAQMv6/e/L6d/duydHHFF1GgBoVUo1ANCyGo9SH3JI0qtXtVkAoJUp1QBAyzH1G4AORqkGAFrOb3+b/OUvycYbl0eqAaCdU6oBgJbTeJT6iCOSnj2rzQIAbUCpBgBaxrJlKy+lZeo3AB2EUg0AtIwHHkhmzUo226y8PjUAdABKNQDQMhqnfh91VHk5LQDoAJRqAGDDLVmSXH99OTb1G4AORKkGADbcr36VzJ2bbLllsv/+VacBgDajVAMAG65x6vfo0UmXLtVmAYA2pFQDABtm0aLkxhvLsanfAHQwSjUAsGFuvz2ZPz9517uSvfeuOg0AtCmlGgDYMI1Tv8eMSTr50wKAjsVvPgCg+V5/PbnllnJs6jcAHZBSDQA036RJycKFybvfnQwbVnUaAGhzSjUA0HyNU7+PPTZpaKg2CwBUQKkGAJrn1VeTX/yiHJv6DUAHpVQDAM1z883J4sXJ3/xNMmRI1WkAoBJKNQDQPKZ+A4BSDQA0w1//mtx1VzkeM6baLABQIaUaAGi6G25Ili1Ldt89ec97qk4DAJVRqgGApnvr1G8A6MCUagCgaWbOTO67rxwfc0y1WQCgYko1ANA0112XFEUyYkQyaFDVaQCgUko1ANA0pn4DwApKNQCw/qZPT37zm6RTp2T06KrTAEDllGoAYP1NnFje77dfsvXWlUYBgFqgVAMA68/UbwBYhVINAKyfJ55Ifve7pEuX5Kijqk4DADVBqQYA1k/j1O+RI5M+farNAgA1QqkGAN5ZUZj6DQBroFQDAO/sd79L/vSnpEeP5Igjqk4DADVDqQYA3lnjUepDDkl69ao2CwDUEKUaAFg3U78BYK2UagBg3X7zm+TZZ5NNNkkOPrjqNABQU5RqAGDdGo9SH3FE0rNntVkAoMYo1QDA2i1bllx7bTk29RsAVqNUAwBrd//9yaxZyWabldenBgBWoVQDAGvXOPX76KOTbt2qzQIANUipBgDWbMmS5Prry7Gp3wCwRko1ALBmv/xl8tJLyVZbJfvtV3UaAKhJSjUAsGaNU79Hj066dKk2CwDUKKUaAFjdm28mN91Ujk39BoC1UqoBgNXdfnsyf36yzTbJiBFVpwGAmqVUAwCra5z6PWZM0smfCwCwNn5LAgCrev31ZNKkcmzqNwCsk1INAKxq0qRk4cJk++2ToUOrTgMANU2pBgBW1Tj1+9hjk4aGarMAQI1TqgGAlV59NfnFL8qxqd8A8I6UagBgpZtvThYvTnbZJRkypOo0AFDzlGoAYKW3Tv0GAN6RUg0AlP761+Suu8rxmDHVZgGAOqFUAwClG25Ili0rz/i9445VpwGAuqBUAwAlU78BoMmUagAgmTkzue++cnzMMdVmAYA6olQDAMl11yVFkey9d7LttlWnAYC6oVQDAKZ+A0AzKdUA0NFNn5785jdJp07Jxz5WdRoAqCtKNQB0dBMnlvf7759svXW1WQCgzijVANDRmfoNAM2mVANAR/bEE8nvfpd06ZIcdVTVaQCg7ijVANCRNU79PuigZIstqs0CAHVIqQaAjqooTP0GgA2kVANAR/Xoo8mf/pT06JEcfnjVaQCgLinVANBRNR6lPuSQpFevarMAQJ1SqgGgIzL1GwBahFINAB3Rr3+dzJiRbLJJcvDBVacBgLqlVANAR9R4lPqII5KePavNAgB1TKkGgI5m2bLk2mvLsanfALBBlGoA6Gjuuy+ZPTvZfPNk5Miq0wBAXVOqAaCjaZz6ffTRSbdu1WYBgDqnVANAR7JkSXL99eXY1G8A2GBKNQB0JHfdlbz8ctKvX7LfflWnAYC6p1QDQEfSOPV79Oikc+dqswBAO9CsUn3JJZdk8ODB6dGjR4YOHZr7779/vbZ78MEH06VLl3zgAx9ozo8FADbEm28mN91Ujk39BoAW0eRSPXHixJx22mk5++yzM23atOy7774ZNWpUZsyYsc7t5s2blxNOOCEf/vCHmx0WANgAv/hFsmBBMnBgMnx41WkAoF1ocqm++OKL8+lPfzqf+cxnsvPOO+fb3/52Bg4cmEsvvXSd2/3TP/1TPv7xj2e4X+IAUI3Gqd9jxiSdfAMMAFpCk36jLl68OFOnTs3It13TcuTIkZkyZcpat7viiivy5z//Oeeee+56/ZxFixZl/vz5q9wAgA3w2mvJpEnl2NRvAGgxTSrVc+fOzbJly9KvX79Vlvfr1y+zZ89e4zZPPfVUzjzzzFx99dXp0qXLev2c8ePHp3fv3ituAwcObEpMAODtJk1K3ngj2WGHZPfdq04DAO1Gs+Z+NTQ0rPK4KIrVliXJsmXL8vGPfzznnXde3vOe96z3648bNy7z5s1bcXvuueeaExMAaNQ49fvYY5M1/M4GAJpn/Q4d/5++ffumc+fOqx2VnjNnzmpHr5NkwYIFeeSRRzJt2rSccsopSZLly5enKIp06dIld955Zw444IDVtuvevXu6d+/elGgAwNq88kp5krLE1G8AaGFNOlLdrVu3DB06NJMnT15l+eTJkzNixIjV1u/Vq1cee+yxPProoytuJ598cnbaaac8+uij2XPPPTcsPQDwzm6+OVmyJBkyJNlll6rTAEC70qQj1UkyduzYHH/88Rk2bFiGDx+eH/zgB5kxY0ZOPvnkJOXU7ZkzZ+bHP/5xOnXqlCFDhqyy/VZbbZUePXqsthwAaCVvnfoNALSoJpfqMWPG5KWXXsr555+fWbNmZciQIbntttsyaNCgJMmsWbPe8ZrVAEAbmTMn+eUvy/GYMdVmAYB2qKEoiqLqEO9k/vz56d27d+bNm5devXpVHQcA6sellyaf+1wybFjy8MNVpwGAurG+PbRZZ/8GAOqEqd8A0KqUagBor55/Prn//nJ8zDHVZgGAdkqpBoD26rrrkqJI9tknGTiw6jQA0C4p1QDQXpn6DQCtTqkGgPbomWeS3/426dQp+djHqk4DAO2WUg0A7dHEieX9AQck/fpVmwUA2jGlGgDaI1O/AaBNKNUA0N48/njy+98nXbsmH/1o1WkAoF1TqgGgvWmc+n3QQckWW1SbBQDaOaUaANqTojD1GwDakFINAO3Jo48mTz6Z9OiRHH541WkAoN1TqgGgPWk8Sn3oocmmm1abBQA6AKUaANoLU78BoM0p1QDQXvz618mMGckmmyQHH1x1GgDoEJRqAGgvGo9SH3lkstFGlUYBgI5CqQaA9mDZsuTaa8uxqd8A0GaUagBoD+67L5k9O9l88+TAA6tOAwAdhlINAO1B49Tvo49OunWrNgsAdCBKNQDUuyVLkuuvL8emfgNAm1KqAaDe3XVX8vLLSb9+yX77VZ0GADoUpRoA6l3j1O9jjkk6d642CwB0MEo1ANSzN99MbrqpHJv6DQBtTqkGgHr2i18kCxYk226b7LVX1WkAoMNRqgGgnjVO/R4zJunk1zoAtDW/fQGgXr32WjJpUjk29RsAKqFUA0C9mjQpeeONZMcdk912qzoNAHRISjUA1KvGqd/HHps0NFSbBQA6KKUaAOrRK6+UJylLTP0GgAop1QBQj26+OVmyJNl11+Rv/qbqNADQYSnVAFCP3jr1GwCojFINAPVmzpzkl78sx2PGVJsFADo4pRoA6s311yfLliUf/GCy/fZVpwGADk2pBoB6Y+o3ANQMpRoA6snzzyf331+Ojzmm2iwAgFINAHXl2mvL+333TbbZptosAIBSDQB1xdRvAKgpSjUA1Is//zl5+OGkU6fkYx+rOg0AEKUaAOrHxInl/Yc/nGy1VbVZAIAkSjUA1A9TvwGg5ijVAFAP/vjH5LHHkq5dk49+tOo0AMD/UaoBoB40Tv3+yEeSzTevNgsAsIJSDQC1rihM/QaAGqVUA0CtmzYteeqpZKONksMPrzoNAPAWSjUA1LrGo9SHHppsskm1WQCAVSjVAFDLli9f+X1qU78BoOYo1QBQy37962TGjGTTTZNRo6pOAwC8jVINALWscer3kUeW36kGAGqKUg0AtWrZsuTaa8uxqd8AUJOUagCoVffem7z4YrLFFsnf/V3VaQCANVCqAaBWNU79PvropFu3arMAAGukVANALVq8OLnhhnJs6jcA1CylGgBq0V13JS+/nPTrl3zoQ1WnAQDWQqkGgFrUOPX7mGOSzp2rzQIArJVSDQC15o03kptvLsemfgNATVOqAaDW/OIXyYIFybbbJnvtVXUaAGAdlGoAqDWNU7/HjEk6+VUNALXMb2oAqCULFiQ//3k5Pu64arMAAO9IqQaAWjJpUvmd6ve8J/nAB6pOAwC8A6UaAGpJ49TvY49NGhqqzQIAvCOlGgBqxSuvJLffXo7HjKk2CwCwXpRqAKgVN92ULFmSvO99yd/8TdVpAID1oFQDQK1469RvAKAuKNUAUAvmzEl++ctybOo3ANQNpRoAasH11yfLlyd77JG8+91VpwEA1pNSDQC1wNRvAKhLSjUAVO3555P77y8voXXMMVWnAQCaQKkGgKr98Ifl/b77Ju96V7VZAIAmUaoBoEp//Wty8cXl+HOfqzYLANBkSjUAVGn8+GTBgmS33ZLRo6tOAwA0kVINAFWZMSP53vfK8fjxSSe/lgGg3vjtDQBVOe+8ZPHiZL/9kpEjq04DADSDUg0AVXjiieTKK8vx+PHlmb8BgLqjVANAFc45J1m+PDnyyGSvvapOAwA0k1INAG3tt79Nbryx/A71hRdWnQYA2ABKNQC0tXHjyvvjj0922aXaLADABlGqAaAt3XVX8qtfJd26JV/5StVpAIANpFQDQFspiuTMM8vxZz+bbLddpXEAgA2nVANAW7nhhmTq1GSTTZKzzqo6DQDQApRqAGgLS5cmZ59djs84I9lqq2rzAAAtQqkGgLZw5ZXJk08mffsmY8dWnQYAaCFKNQC0tjfeWHlSsrPOSnr1qjQOANBylGoAaG2XXJLMnJkMHFieoAwAaDeUagBoTfPmJV/7Wjk+77ykR49q8wAALUqpBoDWdNFFycsvJzvvnBx/fNVpAIAWplQDQGt58cXk4ovL8Ve/mnTpUm0eAKDFKdUA0FouvDBZuDDZY4/kyCOrTgMAtAKlGgBaw/TpyWWXlePx45OGhmrzAACtQqkGgNZw7rnJkiXJgQcmBxxQdRoAoJUo1QDQ0h57LLnqqnLceOZvAKBdUqoBoKWdfXZSFMno0cmwYVWnAQBakVINAC3pwQeTSZOSzp2TCy6oOg0A0MqUagBoKUWRnHlmOf7Up5Kddqo2DwDQ6pRqAGgpv/hF8sADSffuyZe/XHUaAKANKNUA0BKWL0/OOqscn3pqss021eYBANqEUg0ALWHixOR3v0t69Vo5BRwAaPeUagDYUIsXJ+ecU46/9KWkT59q8wAAbUapBoANNWFC8swzSb9+yRe/WHUaAKANKdUAsCFefz05//xy/G//lmyySbV5AIA2pVQDwIb4zneS2bOTwYOTk06qOg0A0MaUagBorldeSb75zXJ8/vlJt27V5gEA2pxSDQDN9Y1vJK++muy6a3LccVWnAQAqoFQDQHPMnJn853+W4699Lencudo8AEAlmlWqL7nkkgwePDg9evTI0KFDc//996913RtvvDEHHnhgttxyy/Tq1SvDhw/PHXfc0ezAAFATLrggefPNZO+9k0MOqToNAFCRJpfqiRMn5rTTTsvZZ5+dadOmZd99982oUaMyY8aMNa5/33335cADD8xtt92WqVOnZv/9989hhx2WadOmbXB4AKjEU08lP/xhOf7615OGhmrzAACVaSiKomjKBnvuuWd23333XHrppSuW7bzzzjnyyCMzfvz49XqNXXbZJWPGjMmXv/zlNT6/aNGiLFq0aMXj+fPnZ+DAgZk3b1569erVlLgA0PKOPTaZODE5+ODk1lurTgMAtIL58+end+/e79hDm3SkevHixZk6dWpGjhy5yvKRI0dmypQp6/Uay5cvz4IFC7LFFlusdZ3x48end+/eK24DBw5sSkwAaD3TppWFOim/Sw0AdGhNKtVz587NsmXL0q9fv1WW9+vXL7Nnz16v1/iP//iPvP766znmmGPWus64ceMyb968FbfnnnuuKTEBoPWcdVZ5//GPJ+9/f7VZAIDKdWnORg1v++5YURSrLVuTa665Jl/5ylfys5/9LFtttdVa1+vevXu6d+/enGgA0HruuSe5/fakS5fyutQAQIfXpFLdt2/fdO7cebWj0nPmzFnt6PXbTZw4MZ/+9Kdz3XXX5e/+7u+anhQAqlQUybhx5fgf/zHZfvtq8wAANaFJ07+7deuWoUOHZvLkyassnzx5ckaMGLHW7a655pp84hOfyH//93/nEJcdAaAe3XJL8utfJz17JuecU3UaAKBGNHn699ixY3P88cdn2LBhGT58eH7wgx9kxowZOfnkk5OU34eeOXNmfvzjHycpC/UJJ5yQ//zP/8xee+214ij3RhttlN69e7fgWwGAVrJs2crvUn/xi0n//tXmAQBqRpNL9ZgxY/LSSy/l/PPPz6xZszJkyJDcdtttGTRoUJJk1qxZq1yz+rLLLsvSpUvz+c9/Pp///OdXLD/xxBNz5ZVXbvg7AIDWdvXVyeOPJ5tvnnzpS1WnAQBqSJOvU12F9b0+GAC0uEWLkp12Sp59NvnGN5RqAOggWuU61QDQ4Vx2WVmoBwxITjml6jQAQI1RqgFgbRYsSC68sByfe255kjIAgLdQqgFgbb71reSvf0123DH55CerTgMA1CClGgDWZO7c5KKLyvEFFyRdu1abBwCoSUo1AKzJ+PHl9O/ddktGj646DQBQo5RqAHi7GTOS732vHI8fn3Ty6xIAWDN/JQDA2513Xnkprf32S0aOrDoNAFDDlGoAeKsnnkiuvLIcjx+fNDRUGgcAqG1KNQC81TnnJMuXJ0cemey1V9VpAIAap1QDQKPf/ja58cby6HTj9akBANZBqQaARmedVd6fcEKyyy7VZgEA6oJSDQBJctddyS9/mXTrlnzlK1WnAQDqhFINAEWRnHlmOf7sZ5Pttqs0DgBQP5RqALjhhmTq1GSTTVZOAQcAWA9KNQAd29Klydlnl+Mzzki22qraPABAXVGqAejYrrwyefLJpE+fZOzYqtMAAHVGqQag43rjjeS888rx2WcnvXpVmwcAqDtKNQAd1yWXJM8/nwwcWJ6gDACgiZRqADqmefOSr32tHJ93XtKjR7V5AIC6pFQD0DFddFHy8svJzjsnxx9fdRoAoE4p1QB0PC++mFx8cTn+6leTLl2qzQMA1C2lGoCO58ILk4ULkz32SI48suo0AEAdU6oB6FimT08uu6wcjx+fNDRUmwcAqGtKNQAdy7nnJkuWJAcemBxwQNVpAIA6p1QD0HE89lhy1VXluPHM3wAAG0CpBqDjOPvspCiS0aOTYcOqTgMAtANKNQAdw4MPJpMmJZ07JxdcUHUaAKCdUKoBaP+KIhk3rhx/8pPJTjtVmwcAaDeUagDav9tvT+6/P+nevTxRGQBAC1GqAWjfli9feZT61FOTbbapNg8A0K4o1QC0bxMnJr/7XdKrV3LmmVWnAQDaGaUagPZr8eLknHPK8Ze+lPTpU20eAKDdUaoBaL8mTEieeSbp1y/54herTgMAtENKNQDt0+uvJ+efX47POSfZZJNq8wAA7ZJSDUD79N3vJrNnJ9ttl/zjP1adBgBop5RqANqfV15JvvGNcnzBBUm3btXmAQDaLaUagPbn619PXn012XXX5Ljjqk4DALRjSjUA7ct3v5v8+7+X4699Lencudo8AEC7plQD0D4URXLWWckXvlCOP//55JBDqk4FALRzXaoOAAAbbOnS8mRkV1xRPr7gguTss5OGhmpzAQDtnlINQH1buDAZMyb5+c+TTp2Syy5LPvOZqlMBAB2EUg1A/XrppeSww5KHHkp69Eh++tPkiCOqTgUAdCBKNQD1acaM5CMfSZ54Itlss2TSpGSffapOBQB0MEo1APXnD38oC/XMmcm73pXccUeyyy5VpwIAOiBn/wagvjzwQLLvvmWh3nnnZMoUhRoAqIxSDUD9+NnPkgMPTF59NRk+vCzY225bdSoAoANTqgGoD//v/yVHHZW8+WZy6KHJXXclW2xRdSoAoINTqgGobUVRXnf6H/8xWb48+dSnkptuSnr2rDoZAIBSDUANW7YsOeWU5MtfLh+fdVbywx8mXZxnEwCoDf4qAaA2vflm8g//kNxwQ9LQkPznfyannlp1KgCAVSjVANSeefOSI45I7r036dYt+clPkmOOqToVAMBqlGoAassLLySjRiW//32y6abJzTcnBxxQdSoAgDVSqgGoHU8+mYwcmTz7bNKvX/KLXyS77VZ1KgCAtXKiMgBqw29/m+y9d1mod9ghmTJFoQYAap5SDUD1br892X//ZO7cZOjQ5MEHk3e/u+pUAADvSKkGoFo/+Uly2GHJwoXJgQcmd9+dbLVV1akAANaLUg1AdS66KDnhhGTp0uTjH09+/vPy5GQAAHVCqQag7S1fnpxxRvIv/1I+Pv308oh1t27V5gIAaCJn/wagbS1enHzqU8nVV5eP//3fk3/+52ozAQA0k1INQNt57bXk6KOTO+9MOndOLr+8nP4NAFCnlGoA2sacOckhhySPPJL07Jlcf30yalTVqQAANohSDUDrmz49GTkyefrppE+f5NZbkz33rDoVAMAGU6oBaF2PPloekZ49Oxk0KLnjjmSnnapOBQDQIpz9G4DWc/fdyYc+VBbq970vmTJFoQYA2hWlGoDWcd11yUc+ksyfn/zt3yb33psMGFB1KgCAFqVUA9Dyvve9ZMyY8vJZRx1VTvnebLOqUwEAtDilGoCWUxTJOeckp5xSjj/72eTaa5MePapOBgDQKpyoDICWsXRpcvLJyYQJ5ePzzy8LdkNDtbkAAFqRUg3Ahlu4MDnuuOSWW5JOnZLvfz856aSqUwEAtDqlGoAN8/LLyWGHlWf27tEjueaa5Mgjq04FANAmlGoAmu+558ozfD/+eHkiskmTkn32qToVAECbUaoBaJ7HH08OOih5/vnkXe9Kbr89GTKk6lQAAG3K2b8BaLopU8oj0s8/n7z3veVjhRoA6ICUagCaZtKk5MMfTl55Jdlrr+SBB5Jtt606FQBAJZRqANbf5ZcnH/1o8uabySGHJL/8ZdKnT9WpAAAqo1QD8M6KIvnqV5NPfzpZtiz55CeTm25KevasOhkAQKWUagDW7c03ky98ITnnnPLxuHHJhAlJ167V5gIAqAHO/g3A6ooieeih5Ec/SiZOTObNSxoakm9/uyzYAAAkUaoBeKtnn01+8pPkxz9Onnpq5fKBA5OLL04+9rHqsgEA1CClGqCje+215IYbyqPSd9+9cnnPnmWJPuGEZP/9k06+MQQA8HZKNUBHtHx5WaB/9KOyUC9cuPK5/fdPTjwxOfroZJNNqssIAFAHlGqAjuTJJ8si/ZOfJM89t3L5DjuURfr445NBg6rLBwBQZ5RqgPbulVfKk4396EfJr3+9cnnv3smYMWWZHj68PBEZAABNolQDtEdLliR33FEW6VtuSRYvLpd37pwcdFBZpA8/POnRo9qcAAB1TqkGaE9+97uySF99dTJnzsrlu+5aFum///tk662rywcA0M4o1QD17sUXyxL94x+XpbrRlluWJfrEE5P3v9/0bgCAVqBUA9SjN99MJk0qj0rffnuybFm5vFu35LDDyiL9kY8kXbtWmxMAoJ1TqgHqRVEkv/lNWaR/+tPk1VdXPrfHHmWRPvbYZIstKosIANDRKNUAtW7GjOSqq8oy/eSTK5dvs015CawTTkje+97q8gEAdGBKNUAteu215MYbyyJ9993lUeok6dkzOeqo8qj0/vuXZ/MGAKAySjVArVi+PLn33rJIX3998vrrK5/bb7/yiPTHPpZsumllEQEAWJVSDVC1p54qi/RPflJO9W60/fblEenjj0+2266yeAAArJ1SDVCFV19NJk4sy/RDD61c3qtXMmZMWaZHjHAZLACAGqdUA7SmN95I/vzn5Omny9tTT5W3KVOSRYvKdTp1Sg46qCzShx+ebLRRtZkBAFhvSjXAhlq4cGVxfuqpVQv088+vfbshQ8oi/fd/n/Tv33Z5AQBoMUo1wPp4/fWyOL+9ND/9dDJz5rq37d072XHHZIcdVt5/4APJrrua3g0AUOeUaoBGr722sjC/tTQ/9VQya9a6t91ss9WLc+N9nz7KMwBAO6VUAx3LggVrLs1PP53Mnr3ubbfYYtWy/PbiDABAh6NUA+3P/PlrLs1PP528+OK6t+3TZ82leYcdylINAABvoVQDtW3ZsvLo8oIFZVmeP3/luPH+1VeTZ55ZWaD/+td1v2bfvqtP0W68bb55m7wtAADaB6UaaHnLl5cn9lpTAV7beG3LXn+9eRm22mr1o8077phsv335/WcAAGgBzSrVl1xySf793/89s2bNyi677JJvf/vb2Xfffde6/r333puxY8fmj3/8YwYMGJAvfelLOfnkk5sdGmgBRVEeBV6yZPXbG2+8c9ldV0F+7bXy9VtS165Jr17JppuW928fDxq0anHu3btlfz4AAKxBk0v1xIkTc9ppp+WSSy7J3nvvncsuuyyjRo3K448/nm233Xa19adPn56DDz44J510Uq666qo8+OCD+dznPpctt9wyRx99dIu8CVijoiiPmDbe3v54fZ/b0G2XL19zca2FW2vr3HnV8vv2+6Ys69699fMCAEATNRRF0w4n7bnnntl9991z6aWXrli2884758gjj8z48eNXW/9f//Vfc8stt+SJJ55Ysezkk0/O7373uzz00ENr/BmLFi3KokWLVjyeP39+Bg4cmHnz5qVXr15Nidu2DjkkmTevHL/9f9YNedwWr1UU6x6v73ptNV6fckvTNTSUR4R79GheCX778xtt5FJSAADUpfnz56d3797v2EObdKR68eLFmTp1as4888xVlo8cOTJTpkxZ4zYPPfRQRo4cucqygw46KBMmTMiSJUvStWvX1bYZP358zjvvvKZEqw2/+U3y0ktVp6C5GhqSTp1W3r/91tTlb32uS5eyrHbtmnTrtnJcxW1dP79z56o/BQAAqCtNKtVz587NsmXL0q9fv1WW9+vXL7PXcn3X2bNnr3H9pUuXZu7cuenfv/9q24wbNy5jx45d8bjxSHXNu/LK1afUvv0oXUs/bsnXbGhYeXvr47WN13e91njtzp2bVmrfaflbXxsAAGA9NetEZQ1vKx9FUay27J3WX9PyRt27d0/3evz+5KGHVp0AAACANtSpKSv37ds3nTt3Xu2o9Jw5c1Y7Gt1o6623XuP6Xbp0SZ8+fZoYFwAAAGpHk0p1t27dMnTo0EyePHmV5ZMnT86IESPWuM3w4cNXW//OO+/MsGHD1vh9agAAAKgXTSrVSTJ27Nj88Ic/zOWXX54nnngip59+embMmLHiutPjxo3LCSecsGL9k08+Oc8++2zGjh2bJ554IpdffnkmTJiQf/7nf265dwEAAAAVaPJ3qseMGZOXXnop559/fmbNmpUhQ4bktttuy6BBg5Iks2bNyowZM1asP3jw4Nx22205/fTT873vfS8DBgzId77zHdeoBgAAoO41+TrVVVjf64MBAABAS1jfHtrk6d8AAABASakGAACAZlKqAQAAoJmUagAAAGgmpRoAAACaSakGAACAZlKqAQAAoJmUagAAAGgmpRoAAACaSakGAACAZlKqAQAAoJmUagAAAGgmpRoAAACaSakGAACAZlKqAQAAoJmUagAAAGgmpRoAAACaSakGAACAZlKqAQAAoJm6VB1gfRRFkSSZP39+xUkAAADoCBr7Z2MfXZu6KNULFixIkgwcOLDiJAAAAHQkCxYsSO/evdf6fEPxTrW7BixfvjwvvPBCNt100zQ0NFQdZ63mz5+fgQMH5rnnnkuvXr2qjkMT+Ozqm8+vfvns6pfPrn757Oqbz69++ezqT1EUWbBgQQYMGJBOndb+zem6OFLdqVOnbLPNNlXHWG+9evWyo9Qpn1198/nVL59d/fLZ1S+fXX3z+dUvn119WdcR6kZOVAYAAADNpFQDAABAMynVLah79+4599xz071796qj0EQ+u/rm86tfPrv65bOrXz67+ubzq18+u/arLk5UBgAAALXIkWoAAABoJqUaAAAAmkmpBgAAgGZSqgEAAKCZlGoAAABoJqW6Cb761a9mxIgR6dmzZzbbbLM1rjNjxowcdthh2XjjjdO3b9984QtfyOLFi9f5uosWLcqpp56avn37ZuONN87hhx+e559/vhXeAY3uueeeNDQ0rPH28MMPr3W7T3ziE6utv9dee7VhcpJku+22W+1zOPPMM9e5TVEU+cpXvpIBAwZko402yn777Zc//vGPbZSYJPnLX/6ST3/60xk8eHA22mijbL/99jn33HPf8d9I+111LrnkkgwePDg9evTI0KFDc//9969z/XvvvTdDhw5Njx498u53vzvf//732ygpjcaPH58PfvCD2XTTTbPVVlvlyCOPzJ/+9Kd1brO234n/+7//20apafSVr3xltc9h6623Xuc29rvasKa/TRoaGvL5z39+jevb79qXLlUHqCeLFy/O6NGjM3z48EyYMGG155ctW5ZDDjkkW265ZR544IG89NJLOfHEE1MURb773e+u9XVPO+20TJo0KT/96U/Tp0+fnHHGGTn00EMzderUdO7cuTXfUoc1YsSIzJo1a5Vl//Zv/5a77rorw4YNW+e2H/nIR3LFFVeseNytW7dWyci6nX/++TnppJNWPN5kk03Wuf43v/nNXHzxxbnyyivznve8JxdeeGEOPPDA/OlPf8qmm27a2nFJ8r//+79Zvnx5Lrvssuywww75wx/+kJNOOimvv/56LrroonVua79rexMnTsxpp52WSy65JHvvvXcuu+yyjBo1Ko8//ni23Xbb1dafPn16Dj744Jx00km56qqr8uCDD+Zzn/tcttxyyxx99NEVvIOO6d57783nP//5fPCDH8zSpUtz9tlnZ+TIkXn88cez8cYbr3PbP/3pT+nVq9eKx1tuuWVrx2UNdtlll9x1110rHq/rb0H7Xe14+OGHs2zZshWP//CHP+TAAw/M6NGj17md/a6dKGiyK664oujdu/dqy2+77baiU6dOxcyZM1csu+aaa4ru3bsX8+bNW+Nrvfrqq0XXrl2Ln/70pyuWzZw5s+jUqVNx++23t3h21mzx4sXFVlttVZx//vnrXO/EE08sjjjiiLYJxVoNGjSo+Na3vrXe6y9fvrzYeuuti69//esrlr355ptF7969i+9///utkJD19c1vfrMYPHjwOtex31Vjjz32KE4++eRVlr33ve8tzjzzzDWu/6Uvfal473vfu8qyf/qnfyr22muvVsvIO5szZ06RpLj33nvXus7dd99dJCleeeWVtgvGGp177rnF+9///vVe335Xu774xS8W22+/fbF8+fI1Pm+/a19M/25BDz30UIYMGZIBAwasWHbQQQdl0aJFmTp16hq3mTp1apYsWZKRI0euWDZgwIAMGTIkU6ZMafXMlG655ZbMnTs3n/jEJ95x3XvuuSdbbbVV3vOe9+Skk07KnDlzWj8gq/nGN76RPn365AMf+EC++tWvrnMK8fTp0zN79uxV9rPu3bvnQx/6kP2sYvPmzcsWW2zxjuvZ79rW4sWLM3Xq1FX2mSQZOXLkWveZhx56aLX1DzrooDzyyCNZsmRJq2Vl3ebNm5ck67Wf7bbbbunfv38+/OEP5+67727taKzFU089lQEDBmTw4ME59thj88wzz6x1XftdbVq8eHGuuuqqfOpTn0pDQ8M617XftQ9KdQuaPXt2+vXrt8qyzTffPN26dcvs2bPXuk23bt2y+eabr7K8X79+a92GljdhwoQcdNBBGThw4DrXGzVqVK6++ur86le/yn/8x3/k4YcfzgEHHJBFixa1UVKS5Itf/GJ++tOf5u67784pp5ySb3/72/nc5z631vUb96W375/2s2r9+c9/zne/+92cfPLJ61zPftf25s6dm2XLljVpn1nT78B+/fpl6dKlmTt3bqtlZe2KosjYsWOzzz77ZMiQIWtdr3///vnBD36QG264ITfeeGN22mmnfPjDH859993XhmlJkj333DM//vGPc8cdd+T//b//l9mzZ2fEiBF56aWX1ri+/a423XzzzXn11VfXebDGfte+dPjvVH/lK1/Jeeedt851Hn744Xf8nm2jNf3XqKIo3vG/UrXENjTv83z++edzxx135Nprr33H1x8zZsyK8ZAhQzJs2LAMGjQot956a4466qjmB6dJn93pp5++Ytn73ve+bL755vnYxz624uj12rx9n7KftYzm7HcvvPBCPvKRj2T06NH5zGc+s85t7XfVaeo+s6b117SctnHKKafk97//fR544IF1rrfTTjtlp512WvF4+PDhee6553LRRRflb//2b1s7Jm8xatSoFeNdd901w4cPz/bbb58f/ehHGTt27Bq3sd/VngkTJmTUqFGrzF59O/td+9LhS/Upp5ySY489dp3rbLfdduv1WltvvXV+85vfrLLslVdeyZIlS1b7r4hv3Wbx4sV55ZVXVjlaPWfOnIwYMWK9fi4rNefzvOKKK9KnT58cfvjhTf55/fv3z6BBg/LUU081eVtWtSH7YuOZoJ9++uk1lurGM6fOnj07/fv3X7F8zpw5a903WX9N/exeeOGF7L///hk+fHh+8IMfNPnn2e9aX9++fdO5c+fVjkqva5/Zeuut17h+ly5d1vkfu2gdp556am655Zbcd9992WabbZq8/V577ZWrrrqqFZLRFBtvvHF23XXXtf57Z7+rPc8++2zuuuuu3HjjjU3e1n5Xvzp8qe7bt2/69u3bIq81fPjwfPWrX82sWbNW/OF+5513pnv37hk6dOgatxk6dGi6du2ayZMn55hjjkmSzJo1K3/4wx/yzW9+s0VydSRN/TyLosgVV1yRE044IV27dm3yz3vppZfy3HPPrVLUaJ4N2RenTZuWJGv9HAYPHpytt946kydPzm677Zak/L7Tvffem2984xvNC8wKTfnsZs6cmf333z9Dhw7NFVdckU6dmv4tJPtd6+vWrVuGDh2ayZMn56Mf/eiK5ZMnT84RRxyxxm2GDx+eSZMmrbLszjvvzLBhw5r17yvNUxRFTj311Nx000255557Mnjw4Ga9zrRp0+xjNWDRokV54oknsu+++67xeftd7bniiiuy1VZb5ZBDDmnytva7OlbZKdLq0LPPPltMmzatOO+884pNNtmkmDZtWjFt2rRiwYIFRVEUxdKlS4shQ4YUH/7wh4v/+Z//Ke66665im222KU455ZQVr/H8888XO+20U/Gb3/xmxbKTTz652GabbYq77rqr+J//+Z/igAMOKN7//vcXS5cubfP32NHcddddRZLi8ccfX+PzO+20U3HjjTcWRVEUCxYsKM4444xiypQpxfTp04u77767GD58ePGud72rmD9/flvG7tCmTJlSXHzxxcW0adOKZ555ppg4cWIxYMCA4vDDD19lvbd+dkVRFF//+teL3r17FzfeeGPx2GOPFccdd1zRv39/n10bmjlzZrHDDjsUBxxwQPH8888Xs2bNWnF7K/tdbfjpT39adO3atZgwYULx+OOPF6eddlqx8cYbF3/5y1+KoiiKM888szj++ONXrP/MM88UPXv2LE4//fTi8ccfLyZMmFB07dq1uP7666t6Cx3SZz/72aJ3797FPffcs8o+tnDhwhXrvP2z+9a3vlXcdNNNxZNPPln84Q9/KM4888wiSXHDDTdU8RY6tDPOOKO45557imeeeab49a9/XRx66KHFpptuar+rE8uWLSu23Xbb4l//9V9Xe85+174p1U1w4oknFklWu919990r1nn22WeLQw45pNhoo42KLbbYojjllFOKN998c8Xz06dPX22bN954ozjllFOKLbbYothoo42KQw89tJgxY0YbvrOO67jjjitGjBix1ueTFFdccUVRFEWxcOHCYuTIkcWWW25ZdO3atdh2222LE0880WfVxqZOnVrsueeeRe/evYsePXoUO+20U3HuuecWr7/++irrvfWzK4ryslrnnntusfXWWxfdu3cv/vZv/7Z47LHH2jh9x3bFFVes8d/Qt//3Xftd7fje975XDBo0qOjWrVux++67r3JZphNPPLH40Ic+tMr699xzT7HbbrsV3bp1K7bbbrvi0ksvbePErG0fe+u/h2//7L7xjW8U22+/fdGjR49i8803L/bZZ5/i1ltvbfvwFGPGjCn69+9fdO3atRgwYEBx1FFHFX/84x9XPG+/q2133HFHkaT405/+tNpz9rv2raEo/u9sBgAAAECTuKQWAAAANJNSDQAAAM2kVAMAAEAzKdUAAADQTEo1AAAANJNSDQAAAM2kVAMAAEAzKdUAAADQTEo1AAAANJNSDQAAAM2kVAMAAEAz/X8uIeoFMIlu6gAAAABJRU5ErkJggg=="
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "execution_count": 5
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "棒极了！现在，我们需要编写代价函数来评估结果。\n",
    "代价函数：\n",
    "$J\\left( \\theta  \\right)=\\frac{1}{m}\\sum\\limits_{i=1}^{m}{[-{{y}^{(i)}}\\log \\left( {{h}_{\\theta }}\\left( {{x}^{(i)}} \\right) \\right)-\\left( 1-{{y}^{(i)}} \\right)\\log \\left( 1-{{h}_{\\theta }}\\left( {{x}^{(i)}} \\right) \\right)]}$"
   ]
  },
  {
   "cell_type": "code",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2024-08-07T02:04:38.224490Z",
     "start_time": "2024-08-07T02:04:38.220836Z"
    }
   },
   "source": [
    "def cost(theta, X, y):\n",
    "    theta = np.matrix(theta)\n",
    "    X = np.matrix(X)\n",
    "    y = np.matrix(y)\n",
    "    first = np.multiply(-y, np.log(sigmoid(X * theta.T)))\n",
    "    second = np.multiply((1 - y), np.log(1 - sigmoid(X * theta.T)))\n",
    "    return np.sum(first - second) / (len(X))"
   ],
   "outputs": [],
   "execution_count": 6
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "现在，我们要做一些设置，和我们在练习1在线性回归的练习很相似。"
   ]
  },
  {
   "cell_type": "code",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2024-08-07T02:04:38.259123Z",
     "start_time": "2024-08-07T02:04:38.253070Z"
    }
   },
   "source": [
    "# add a ones column - this makes the matrix multiplication work out easier\n",
    "data.insert(0, 'Ones', 1)\n",
    "\n",
    "# set X (training data) and y (target variable)\n",
    "cols = data.shape[1]\n",
    "X = data.iloc[:,0:cols-1]\n",
    "y = data.iloc[:,cols-1:cols]\n",
    "\n",
    "# convert to numpy arrays and initalize the parameter array theta\n",
    "X = np.array(X.values)\n",
    "y = np.array(y.values)\n",
    "theta = np.zeros(3)"
   ],
   "outputs": [],
   "execution_count": 7
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "让我们来检查矩阵的维度来确保一切良好。"
   ]
  },
  {
   "cell_type": "code",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2024-08-07T02:04:38.276263Z",
     "start_time": "2024-08-07T02:04:38.270610Z"
    }
   },
   "source": [
    "theta"
   ],
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([0., 0., 0.])"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "execution_count": 8
  },
  {
   "cell_type": "code",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2024-08-07T02:04:38.300304Z",
     "start_time": "2024-08-07T02:04:38.288603Z"
    }
   },
   "source": [
    "X.shape, theta.shape, y.shape"
   ],
   "outputs": [
    {
     "data": {
      "text/plain": [
       "((100, 3), (3,), (100, 1))"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "execution_count": 9
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "让我们计算初始化参数的代价函数(theta为0)。"
   ]
  },
  {
   "cell_type": "code",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2024-08-07T02:04:38.345160Z",
     "start_time": "2024-08-07T02:04:38.339884Z"
    }
   },
   "source": [
    "cost(theta, X, y)"
   ],
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.6931471805599453"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "execution_count": 10
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "看起来不错，接下来，我们需要一个函数来计算我们的训练数据、标签和一些参数thata的梯度。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# gradient descent(梯度下降)\n",
    "* 这是批量梯度下降（batch gradient descent）  \n",
    "* 转化为向量化计算： $\\frac{1}{m} X^T( Sigmoid(X\\theta) - y )$\n",
    "$$\\frac{\\partial J\\left( \\theta  \\right)}{\\partial {{\\theta }_{j}}}=\\frac{1}{m}\\sum\\limits_{i=1}^{m}{({{h}_{\\theta }}\\left( {{x}^{(i)}} \\right)-{{y}^{(i)}})x_{_{j}}^{(i)}}$$"
   ]
  },
  {
   "cell_type": "code",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2024-08-07T02:04:38.401812Z",
     "start_time": "2024-08-07T02:04:38.397667Z"
    }
   },
   "source": [
    "def gradient(theta, X, y):\n",
    "    theta = np.matrix(theta)\n",
    "    X = np.matrix(X)\n",
    "    y = np.matrix(y)\n",
    "    \n",
    "    parameters = int(theta.ravel().shape[1])\n",
    "    grad = np.zeros(parameters)\n",
    "    \n",
    "    error = sigmoid(X * theta.T) - y\n",
    "    \n",
    "    for i in range(parameters):\n",
    "        term = np.multiply(error, X[:,i])\n",
    "        grad[i] = np.sum(term) / len(X)\n",
    "    \n",
    "    return grad"
   ],
   "outputs": [],
   "execution_count": 11
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "注意，我们实际上没有在这个函数中执行梯度下降，我们仅仅在计算一个梯度步长。在练习中，一个称为“fminunc”的Octave函数是用来优化函数来计算成本和梯度参数。由于我们使用Python，我们可以用SciPy的“optimize”命名空间来做同样的事情。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "我们看看用我们的数据和初始参数为0的梯度下降法的结果。"
   ]
  },
  {
   "cell_type": "code",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2024-08-07T02:04:38.422619Z",
     "start_time": "2024-08-07T02:04:38.417317Z"
    }
   },
   "source": [
    "gradient(theta, X, y)"
   ],
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([ -0.1       , -12.00921659, -11.26284221])"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "execution_count": 12
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "现在可以用SciPy's truncated newton（TNC）实现寻找最优参数。"
   ]
  },
  {
   "cell_type": "code",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2024-08-07T02:04:38.823539Z",
     "start_time": "2024-08-07T02:04:38.433110Z"
    }
   },
   "source": [
    "import scipy.optimize as opt\n",
    "result = opt.fmin_tnc(func=cost, x0=theta, fprime=gradient, args=(X, y))\n",
    "result"
   ],
   "outputs": [
    {
     "ename": "ModuleNotFoundError",
     "evalue": "No module named 'scipy'",
     "output_type": "error",
     "traceback": [
      "\u001B[1;31m---------------------------------------------------------------------------\u001B[0m",
      "\u001B[1;31mModuleNotFoundError\u001B[0m                       Traceback (most recent call last)",
      "Cell \u001B[1;32mIn[13], line 1\u001B[0m\n\u001B[1;32m----> 1\u001B[0m \u001B[38;5;28;01mimport\u001B[39;00m \u001B[38;5;21;01mscipy\u001B[39;00m\u001B[38;5;21;01m.\u001B[39;00m\u001B[38;5;21;01moptimize\u001B[39;00m \u001B[38;5;28;01mas\u001B[39;00m \u001B[38;5;21;01mopt\u001B[39;00m\n\u001B[0;32m      2\u001B[0m result \u001B[38;5;241m=\u001B[39m opt\u001B[38;5;241m.\u001B[39mfmin_tnc(func\u001B[38;5;241m=\u001B[39mcost, x0\u001B[38;5;241m=\u001B[39mtheta, fprime\u001B[38;5;241m=\u001B[39mgradient, args\u001B[38;5;241m=\u001B[39m(X, y))\n\u001B[0;32m      3\u001B[0m result\n",
      "\u001B[1;31mModuleNotFoundError\u001B[0m: No module named 'scipy'"
     ]
    }
   ],
   "execution_count": 13
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "让我们看看在这个结论下代价函数计算结果是什么个样子~"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.20349770158947458"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "cost(result[0], X, y)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "接下来，我们需要编写一个函数，用我们所学的参数theta来为数据集X输出预测。然后，我们可以使用这个函数来给我们的分类器的训练精度打分。\n",
    "逻辑回归模型的假设函数： \n",
    "\t\\\\[{{h}_{\\theta }}\\left( x \\right)=\\frac{1}{1+{{e}^{-{{\\theta }^{T}}X}}}\\\\] \n",
    "当${{h}_{\\theta }}$大于等于0.5时，预测 y=1\n",
    "\n",
    "当${{h}_{\\theta }}$小于0.5时，预测 y=0 。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [],
   "source": [
    "def predict(theta, X):\n",
    "    probability = sigmoid(X * theta.T)\n",
    "    return [1 if x >= 0.5 else 0 for x in probability]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "accuracy = 89%\n"
     ]
    }
   ],
   "source": [
    "theta_min = np.matrix(result[0])\n",
    "predictions = predict(theta_min, X)\n",
    "correct = [1 if ((a == 1 and b == 1) or (a == 0 and b == 0)) else 0 for (a, b) in zip(predictions, y)]\n",
    "accuracy = (sum(map(int, correct)) % len(correct))\n",
    "print ('accuracy = {0}%'.format(accuracy))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "我们的逻辑回归分类器预测正确，如果一个学生被录取或没有录取，达到89%的精确度。不坏！记住，这是训练集的准确性。我们没有保持住了设置或使用交叉验证得到的真实逼近，所以这个数字有可能高于其真实值（这个话题将在以后说明）。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 正则化逻辑回归"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "在训练的第二部分，我们将要通过加入正则项提升逻辑回归算法。如果你对正则化有点眼生，或者喜欢这一节的方程的背景，请参考在\"exercises\"文件夹中的\"ex2.pdf\"。简而言之，正则化是成本函数中的一个术语，它使算法更倾向于“更简单”的模型（在这种情况下，模型将更小的系数）。这个理论助于减少过拟合，提高模型的泛化能力。这样，我们开始吧。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "设想你是工厂的生产主管，你有一些芯片在两次测试中的测试结果。对于这两次测试，你想决定是否芯片要被接受或抛弃。为了帮助你做出艰难的决定，你拥有过去芯片的测试数据集，从其中你可以构建一个逻辑回归模型。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "和第一部分很像，从数据可视化开始吧！"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Test 1</th>\n",
       "      <th>Test 2</th>\n",
       "      <th>Accepted</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0.051267</td>\n",
       "      <td>0.69956</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>-0.092742</td>\n",
       "      <td>0.68494</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>-0.213710</td>\n",
       "      <td>0.69225</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>-0.375000</td>\n",
       "      <td>0.50219</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>-0.513250</td>\n",
       "      <td>0.46564</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "     Test 1   Test 2  Accepted\n",
       "0  0.051267  0.69956         1\n",
       "1 -0.092742  0.68494         1\n",
       "2 -0.213710  0.69225         1\n",
       "3 -0.375000  0.50219         1\n",
       "4 -0.513250  0.46564         1"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "path =  'ex2data2.txt'\n",
    "data2 = pd.read_csv(path, header=None, names=['Test 1', 'Test 2', 'Accepted'])\n",
    "data2.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAuMAAAHjCAYAAACJlRE5AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvOIA7rQAAIABJREFUeJzs3X185HV57//3FVCRbKjcqZgF2Z7dVrTAskYqsjXFG6BbC2EFs3hHdVs8ao+77MN2l9pTTj160LVtjC2t5awoWgurnGygj66VW392RdTQglJAEhR0d0EoaDsJhQq5fn985ku+mcwkM8nM9/b1fDzymMxnJslnvvkmc81nrs91mbsLAAAAQPK60p4AAAAAUFYE4wAAAEBKCMYBAACAlBCMAwAAACkhGAcAAABSQjAOAAAApIRgHAAAAEgJwTgAAACQEoJxAAAAICUHpj2BJB1xxBF+7LHHpj0NAAAAFNjtt9/+b+5+ZDP3LVUwfuyxx2psbCztaQAAAKDAzOzBZu9LmgoAAACQEoJxAAAAICUE4wAAAEBKSpUzDgAAgODnP/+59u7dqyeffDLtqeTWQQcdpOXLl+s5z3nOor8HwTgAAEAJ7d27Vz09PTr22GNlZmlPJ3fcXY899pj27t2rFStWLPr7kKYCAABQQk8++aQOP/xwAvFFMjMdfvjhS35ngWAcAACgpAjEl6Ydx49gHAAAAEgJwTgAAAAWVKlIO3ZIW7eGy0qlPd93dHRUZqZ77723Pd9wAZ/85Cf1xBNPtPQ1X/va1/SmN72pI/MhGAcAAMC89uyRenulzZul7dvDZW9vGF+qq666SmvXrtVVV1219G/WhMUE451EMA4AAICGKhVp3bpwOTUVxqamZsYnJxf/vScnJ7Vnzx595jOf0dVXX/3s+Mc//nEdf/zxOvHEE7Vt2zZJ0sTEhN7whjfoxBNP1Jo1a3T//fdLkj7xiU/oVa96lU444QRdcsklkqQHHnhAL3vZy/S2t71Nxx13nM4991w98cQT+tSnPqX9+/frtNNO02mnnSZJuv7663XKKadozZo1Ou+88zRZfUD/+I//qJe97GVas2aNRkZGFv8gF0AwDgAAgIZ27pSmp+vfNj0dbl+sa6+9VmeeeaZ+6Zd+SYcffrhuv/12feUrX9G1116rb33rW7rzzjv1B3/wB5Kkt73tbXr/+9+vO++8U7feequOOuooXX/99RofH9e3v/1t3XHHHbr99tv19a9/XZL0/e9/X+973/t0zz336JBDDtFf/dVf6QMf+IBe8pKX6JZbbtEtt9yif/u3f9NHPvIR3Xjjjfrnf/5n9fX16c///M/15JNP6nd/93f193//97r99tv18MMPL/5BLoBgHAAAAA2Nj8+siNeampImJhb/va+66ipt2LBBkrRhwwZdddVVuvHGG/Wud71LBx98sCTpsMMOU6VS0b59+3TOOedICs12Dj74YF1//fW6/vrrddJJJ2nNmjW69957NT4+Lkk6+uijdeqpp0qS3v72t2tPnZya2267TXfffbdOPfVUrV69WldeeaUefPBB3XvvvVqxYoVWrVolM9Pb3/72xT/IBdD0BwAAAA2tWiV1d9cPyLu7pZUrF/d9H3/8cd1888363ve+JzPTM888IzPTeeed1/T3cHddfPHFes973jNr/IEHHphTdrBeGUJ31xvf+MY5+ep33HFHC49kaVgZBwAAQEODg1JXg4ixqyvcvhjXXHON3vGOd+jBBx/UAw88oB//+MdasWKFfuEXfkGf/exnn91k+fjjj6unp0fLly/X6OioJOmpp57SE088oTPOOENXXHHFs3ne+/bt0yOPPCJJ+tGPfqRvfvObkqS/+7u/09q1ayVJPT09qlRLwbz61a/WN77xDU1Ul/enpqZ033336WUve5keeOCBZ/PSO7m5lGAcAAAADfX0SLt3h8vu7jDW3T0zvmzZ4r7vVVdd9WzaSeTNb36zHnroIZ111lnq6+vT6tWr9ad/+qeSpC984Qv61Kc+pRNOOEGvec1r9PDDD+v000/XW9/6Vp1yyik6/vjjde655z4baP/yL/+yLrvsMh133HH66U9/qve+972SpAsvvFBnnnmmTjvtNB155JH63Oc+p/PPP18nnHCCTjnlFN1777066KCDdPnll+s3f/M3tWbNGr3whS9c3INsgrl7x7551vT19fnY2Fja0wAAAEjdPffco+OOO67p+09Ohs2aExMhNWVwcPGBeKc98MADetOb3qS77rqr4z+r3nE0s9vdva+ZrydnHABquUujo9LAgBTPMWw0DgAlsGyZtHFj2rMoHtJUAKDW6Ki0fr100UUhAJfC5UUXhfFqziIAIJuOPfbYRFbF24GVcQCoNTAgbdokDQ+H60NDIRAfHg7jAwPpzg8AUBgE4wBQyywE4FIIwKOgfNOmME6KCgCgTUhTAYB64gF5hEA8v9ylXbtm0o4WGgeAhBCMA0A9UY54XDyHHPnCPgAAGUUwDgC1oiAtyhGfnp7JIScgz6f4PoDod8g+AKA5HXxn6YADDtDq1av1K7/yK/qt3/ot/exnP1vwa17zmtcs6meNjo7q7rvvbvnrlnW4fiPBOADUGh2dCdKi1JShoZlgjlXU/Kn9HXZ1zf0dA6ivg+8sPf/5z9cdd9yhu+66S4cddpguu+yyBb/m1ltvXdTPWmww3mkE4wBQa2BAGhmZHaRFwdzICKuoecU+AGBxEnpn6ZRTTtG+ffuevf6JT3xCr3rVq3TCCSfokksueXY8vlLd6D6f//zndcIJJ+jEE0/UO97xDt1666267rrr9Pu///tavXq17r//ft1///0688wz9cpXvlK/9mu/pnvvvVeS9MMf/vDZjp5/9Ed/1JbHNh+qqQBALTOppkXzvOPIh0b7AAjIgfklUGHqmWee0U033aSN1a5C119/vcbHx/Xtb39b7q6zzjpLX//61/Xa17722a9pdJ/DDz9cH/nIR3TrrbfqiCOO0OOPP67DDjtMZ511lt70pjfp3HPPlSS9/vWv16c//WmtWrVK3/rWt/S+971PN998szZt2qT3vve9euc739nUSv1SsTIOACg+9gEAS9Ohd5b+8z//U6tXr9aLX/xi/eQnP9Eb3/hGSSHQvv7663XSSSdpzZo1uvfeezU+Pj7raxvd5+abb9Z5552nI444QpJ02GGHzfm5k5OTuvXWW3Xeeedp9erVes973qOHHnpIkvSNb3xD559/viTpHe94x5IeXzNYGQcAFF+jfQBSGO/v510PYD4demcpyhl/4okndMYZZ+iyyy7TBz7wAbm7Lr74Yr3nPe+ZZ0r17/MXf/EXC/7c6elpveAFL9Add9xR93ZL8N0yVsYBAMXHPgBg8RJ4Z+nggw/Wpz71Kf3Zn/2Znn76aZ1xxhm64oorNDk5KUnat2+fHnnkkVlf0+g+r3vd6/TlL39Zjz32mCTp8ccflyT19PSoUqlIkg455BCtWLFCX/7yl6sP0XXnnXdKkk499VRdffXVkqQvfvGLS35sCyEYBwAUX5TvX7va1WgcwIyEKkyddNJJOuGEE3TVVVfp9NNP11vf+tZnN1Kee+65zwbS0ap1o/u84hWv0Ic+9CH19/frxBNP1JYtWyRJGzZs0Cc+8QmddNJJuv/++/XFL35Rn/nMZ3TiiSfqFa94ha699lpJ0vDwsC677DIdf/zxszaUdop5ifLk+vr6fGxsLO1pAAAApO6ee+7Rcccdt/Ad3UPAPTAw+4Vro/EOeuyxx7RmzRo9+OCDify8ZtQ7jmZ2u7v3NfP1rIwDAACgsYy8s7R//36dcsop+uAHP5jIz0sKGzgBAACQeS95yUt03333pT2NtmNlHAAAoKTKlK7cCe04fgTjAAAAJXTQQQfpscceIyBfJHfXY489poMOOmhJ34c0FQAAgBJavny59u7dq0cffTTtqeTWQQcdpOXLly/pexCMAwAAlNBznvMcrVixIu1plB5pKgAAAEBKCMaBetylXbvmdhVrNA40i3MLABCTajBuZleY2SNmdleD283MPmVmE2b2XTNbE7vtAjMbr35ckNysUQqjo9L69bPb/EbtgNevb1u3MZQQ5xYAICbtlfHPSTpzntt/Q9Kq6seFkv5akszsMEmXSPpVSSdLusTMDu3oTFEuAwMzbX6joOmii2baAQ8MpD1DLEWaq9OcWwCAmFSDcXf/uqTH57nL2ZI+78Ftkl5gZkdJOkPSDe7+uLv/VNINmj+oB1pjJg0NzQRNXV0zwdLQUGLdxtAhaa5Oc24BAGLSXhlfSK+kH8eu762ONRqfw8wuNLMxMxujdA9aEgVNcQRLxZD26jTnFgCgKuvB+JK5++Xu3ufufUceeWTa00GeRAFaXHwlFfmV9uo05xYAoCrrwfg+SUfHri+vjjUaB9qjdqV0enruSiryLa3Vac4tAEBM1oPx6yS9s1pV5dWS/t3dH5L0VUmnm9mh1Y2bp1fHgPYYHZ27UhpfSaXiRf6ltTrNuQUAiEm1A6eZXSXp1yUdYWZ7FSqkPEeS3P3TknZLWidpQtITkt5Vve1xM/vfkr5T/VYfdvf5NoICrRkYkEZGwmW0UhoFTf39pa54UalIO3dK4+PSqlXS4KDU05P2rFpUuzo9NDRzXersCjnnFgAgxrxEb4n29fX52NhY2tMAcmvPHmndupBZMTUldXeHdOvdu6W1a9OeXQt27QpVU+Kr0/EAfWREOuectGcJAMgpM7vd3fuaui/BOIBmVCpSb2+4rNXTI+3fLy1blvy8FsU9pIPEV6fnGwcAoAWtBONZzxkHkBE7d4YV8Xqmp8PtuWEWVr5rA+5G4wAAdAjBOICmjI+H1JR6pqakiYlk5wMAQBEQjANoyqpVIUe8nu5uaeXKZOcDAEAREIwDaMrgYNisWU9XV7gdqMs9bJqt3aPUaBwASoRgHEBTenpC1ZSenpkV8u7umfHcbN5E8kZHQ/WaeB33qHrN+vXUVgdQaqnWGQeQL2vXhqopO3eGHPGVK8OKOIE45jUwMNPUSJpd133TJmqrAyg1ShsCWUTpPTQrL+dKvI57JF7nHQAKhNKGQN7xtj6alZdzJeoyGkcgDgAE40Amxd/Wj4Is3tZHPXk5V6J5xcVfQABASRGMo5yyXt0hWkWMgqyurpngitVExOXhXKl9gTA9PfcFBACUFDnjKKddu8Jb+PGAJR4wjIyEToxpc59dT3B6OhvBFbIny+dKXv7eAKBNyBkHFpKHt/Z5Wx/Nyvq5MjAQAu74Sn20oj8yko2/NwBICcE4yinrb+3ztj6alYdzxSysfNf+XTUaB+KynlYILBHBOMory9UdRkfnvjiIv3jISoUMpI9zBUWXl4pBwCIRjKO8svzWPm/ro1mcKyi6PKQVAkvABk6UU+0/89qOgFlZIQcA0DQKudPKBk6CcZQT1R0AIF+yXDEIqEE1FWAhvLUPAPmR5bRCYIkIxlFOVHcAgHzIQ8UgYAkOTHsCAAAADTWqGCSF8f5+0gqRawTjAAAgu6K0woGBuWmF/f2kFSL3CMYBAEB2RemDzY4DOUPOOAAAAJASgnEAAAAgJQTjAAAAQEoIxgEAAICUEIwDAJAU99ABuLY2dqNxAIVHMA4AbVCpSDt2SFu3hstKJe0ZIZNGR6X162c3q4ma2qxfH24HUCqUNgSAJdqzR1q3LjQGnJqSurulLVuk3bultWvTnh0yZWBgpnukFGplx7tLUjMbKB3zEr0l1tfX52NjY2lPA0CBVCpSb2/9lfCeHmn/fmnZsuTnhQyLt3ePxLtLAsg9M7vd3fuauS9pKgByJ0spITt3hhXxeqanw+3ALPF27hECcaC0CMYB5MqePWElevNmafv2cNnbG8bTMD4eUlPqmZqSJiaSnQ9yIFoZj4vnkAMoFYJxALlRqYTc7EplJgCempoZn5xMfk6rVoUc8Xq6u6WVK5OdDzIunqKyaVN4+yTKIScgB0qJYBxAbmQxJWRwUOpq8J+0qyvcDjxrdHQmEI9SU4aGZgJyqqkApUMwDiA3spgS0tMTqqb09MyskHd3z4yzeROzDAxIIyOzc8SjgHxkhGoqQAkRjAN4VpY2RtaT1ZSQtWtD1ZThYWnbtnC5f38OyhpmuQFNlue2FGbSOefM3azZaBxA4RGMA5CUvY2R9WQ5JWTZMmnjRunSS8NlLlbEs9yAJstzA4A2IhgHkMmNkfWQEtJm8QY0UdCblQY0WZ4bALQRHTgBNLUxcuPGZOfUSJQSsnNnyBFfuTKsiBOIL0K83vXw8EwTmiw0oMny3ACgjejACUBbt4bUlEa2bQvpFygo99n5P9PT2Ql2szy3vHMP6T4DA7OPaaNxAE2jAyeAlmR1YyQSkOUGNFmeWxGQlw9kAsE4gExvjEQHZbkBTZbnVhTk5QOZkGrOuJmdKWlY0gGSdrj7x2puH5J0WvXqwZJe6O4vqN72jKTvVW/7kbuflcysgeKJNkCuWxdinqmpsCLe1cXGyEJr1IBGCuP9/aHcHnMrJvLygUxILWfczA6QdJ+kN0raK+k7ks5397sb3P9/SDrJ3d9dvT7p7i2FCOSMA/ObnGRjZKlkOWc4y3MrGvLygbZrJWc8zZXxkyVNuPsPJMnMrpZ0tqS6wbik8yVdktDcgFKKamWjJKJGM82OJynLcyuSRnn5rIwDiUkzZ7xX0o9j1/dWx+Yws5dKWiHp5tjwQWY2Zma3mVnDxDYzu7B6v7FHH320HfMGACD/spSXX9SOq0AT8rKBc4Oka9z9mdjYS6vL/2+V9Ekz+2/1vtDdL3f3PnfvO/LII5OYKwAA2dcoLz8KyJOspkJlF5RYmmkq+yQdHbu+vDpWzwZJ748PuPu+6uUPzOxrkk6SdH/7pwkAQAENDEgjI7Pz76OAvL8/2Woq8couUpgDlV1QEmlu4DxQYQPn6xWC8O9Iequ7/2vN/V4m6R8lrfDqZM3sUElPuPtTZnaEpG9KOrvR5s8IGzgBAMioeNpMhMouyKlcNP1x96cl/Z6kr0q6R9KX3P1fzezDZhYvU7hB0tU++1XDcZLGzOxOSbdI+thCgTgAAMiweKnFCIE4SiDVnHF33+3uv+Tu/83dP1od+2N3vy52n//l7ttqvu5Wdz/e3U+sXn4m6bkjQ/K68Sev8waATqDjKkoqLxs4gcbyuvEnr/NOQaUi7dghbd0aLiuVtGeULRwf5F6WKrsASXP30ny88pWvdBTQ9LT7pk3uUrisdz2L8jrvhP3TP7n39Lh3d4dD090drv/TP6U9s2zg+KAQRkbm/u+L/08cGUl3fkCLJI15k/Fpahs408AGzgLL68afvM57iSqV0OlzfFxatSp0+uzpqX+/3t76K709PdL+/eXuEMrx6axmz1O0AR1XUTCtbOAkGEdx5LWlc17nvUh79kjr1oWHOTUldXeHh797t7R27ez77tghbd4c7leruzu8hilzx1COT+e0cp4CQK1cVFNBxuR9M2FeN/7kdd6LVKmEAKdSmQkgp6ZmxicnZ99/fLx+oBl93cRE++eXp9zrpI9PWbR6ngLAUhCMI8jzZsK8bvzJ67yXYOfO8DDrmZ4Ot8etWhVWJOvp7pZWrmzf3PbsCSkfmzdL27eHy97eMJ5VSR6fMmn1PAWApSAYRxDvfhYFgnnpfpalls6tyOu8l6DVldzBwdkZPHFdXeH2dsjrSmhSx6dseMcBQJIOTHsCyIh4s4Xh4ZkNhXnYTJills6tyOu8lyBayW2U41y7ktvTE3J0G+XutmtzYjMroVnMvU7q+JRNq+dp0thYChQLGzgxW8k2EyJZi63+MTkZgo+JiRAIDQ62N9DcujWkpjSybZt06aXt+3nt1unjUzZZrlLDxtKCo6pMYbSygZOVccxotJkw6yvjyI3FruQuW9bZlemsr4QupNPHp2yy+o5DPJ0qEp2z69ZRyrIQov1b8Xel42mjIyPSOeekPUu0GcE4gtoc8aGh2fWvCcjRJmvXhqAhSyu5g4PSli31byP3upyyeJ7mNZ0KLYjv35JmPxdnff8WFo1gHEGjzYRSGO/v59U42qbVldxO58hmdSUU6craOw5sLC2BPO/fwqKRM46APDVkVJI5suReI8to8lQi7N/KPTpwNkAwDuRLljfSAUnj76EJRVhYiqeNRlgZzx06cAJtlLeujEVC8xVgRpRO1dMz0+ypu3tmvPSBuJTvBnZSKZvBgZxxYF71UiS2bKGMWFLIkQVmy+LG0kzJ+wZI9m+VEsE40ABlxNKX95KDQCdkbWNppuR9A2QJm8GBNBWgIVIk0ke7d3SEu7Rr19y3/BuNI1/iAXkkD4G4FOZ4zjlz59poHIVAMA40QIpE+siRRUfkPa8Y82vUwI4XWcgo0lSABkiRyAZyZNF2ec8rRmM0sEMOUdoQaIAyYkCBUT6umHbtop08MoE64w0QjKNVSTacAZAwGqsUTxHqjKMQqDOOXMlyHe8oRWJ4WNq2LVzu308gDuQeecXp6tQmWjZAIofIGUeq8lDHmzJiQMGQV5y+aBMt6SQAwTjSQx1vAKmgsUr62EQLPItgHKlppo43K9IA2q7JxiqVSvg/ND4eqisNDobN22iDvDfnAdqIDZxIzdat0vbtjW/ftk269NLk5gNgYWUJUNm8nRA20aKg2MCJXIjqeNdDHW8ge/bsCeU+N28OL6Q3bw7X9+xJe2btFU+hi1LnpqZmxicn051fYbCJFpBEMI4U0eocyI8yBajNpNBhiWo30U5Pz+SQE5CjZAjGkRpanQP5UaYAdXy8fuddKYxPTCQ7n0JqtIk2CshHR9OeIZAYNnAiVbQ6B/KhTAFqlEJX7/GSQtcmTW6iBcqAYBypo443sqIsmxMXo0wB6uBg6HdQDyl0bRI14Wl2HCgwqqkAKJTFBtRUz5hfpRI2a9brkNvTU7y+AJwPAJailWoqBOMACmOxAVTZAs3FKluAOjlJCh2AxWklGCdNpUzcw6aYeI7efONAjiyloysNqJrT7B6PoqT7kEIHIAkE42UyOiqtXz9793q8vNTICLl6yK2lBNRl2py4VAsFqPVWz7dsKe7qOQAsFaUNy2RgYG4d13idV3avI8eWElDTgKo9ylSLHADahWC8TGrruHZ1za3zCuTUUgJqGlC1RydqkVcq0o4d0tat4bJeXj8A5BnBeNlEAXkcgTgKYCkBNQ2o2qPd6T579oSNtZs3S9u3h8ve3jAOAEVBMF42UWpKHK2HUQBLDaijzYnDw9K2beFy/37ynFvRznQfUl4AlAUbOMukNkd8aGjmusQKOXJvqR1dqZ6xNO1slkOFGwBlQTBeJqOjc3PEo5SV4eHQgphqKplUlFJxSSCgTk/0LkSjWuStpPtQ4Qa5QdlgLBHBeJkMDITyhfF/DFFA3t9PNZWMolQc8mSp705EopSXegE5FW6QKZQNxhKl2oHTzM6UNCzpAEk73P1jNbf/tqRPSNpXHfpLd99Rve0CSX9UHf+Iu1+50M+jA2fGsbowB50hUVac+8iN+VJAqVZWWq104ExtA6eZHSDpMkm/Ienlks43s5fXuetOd19d/YgC8cMkXSLpVyWdLOkSMzs0oamjU6LVhfiG0uif3Pr14faS6USpOCAPqHCTbZScjKFsMJYozTSVkyVNuPsPJMnMrpZ0tqS7m/jaMyTd4O6PV7/2BklnSrqqQ3NFEuJNiaS5qwslTKMhbxZl1q6UF7QXqXN1RAF59PwlEYijaWkG472Sfhy7vldhpbvWm83stZLuk3SRu/+4wdf21vshZnahpAsl6ZhjjmnDtNExtRtKo39qJV5dIG8WZceG3GyJl5yMRP+f1q0rcfpQo7LBJX3uQmuyXmf87yUd6+4nSLpB0oJ54bXc/XJ373P3viOPPLLtE0Sb0ZRolrQ7Q/JWNIA4UufqqM0Zn56eeZeXPh5oQprB+D5JR8euL9fMRk1Jkrs/5u5PVa/ukPTKZr8WOUVTolnSzJul+yGAWqTO1dGobHAUkJdwvxNak2aaynckrTKzFQqB9AZJb43fwcyOcveHqlfPknRP9fOvSvo/sU2bp0u6uPNTRkcl2JQoT3W708ib5a1oAPWQOlcHZYOxRGmXNlwn6ZMKpQ2vcPePmtmHJY25+3VmdqlCEP60pMclvdfd761+7bsl/WH1W33U3T+70M+jtGHG7dqVSK3WepuPoqYkpd18VGPHjrAS3ugJd3iYPF6gjCg5CTSnldKGqQbjSSMYz7gE6ozzRNKcrVtDakoj27ZJl16a3HwAZAcLGsDCWgnG6cCJ7DCrv/LdaHwRmtl8xIovb0UDaIySk0B7EYyjVNh81JzBwVA3uJ4kqrgAyDZKTgLtQzCOUmHFtzlRtZZGb0WzAgag6PK00R/5Rs44SoWc8dZMTvJWNIDyIS8eS0XOONAAK76t4a1oAGVDaVckjWAcpcPmIwBAI2z0R9IIxlFKrPgCAOphoz+S1pX2BAAAALIi2uhfDxv90QkE4wAAAFWDg2EfUT2UdkUnEIwDAABURRv9e3pmVsi7u2fG2V+EdiNnHAAAIIaN/kgSwTgApICGIkC2sdEfSSEYB4CE1WsosmULDUUAoIzIGQeABMUbikTl06amZsYnJ9OdHwAgWQTjAJCgZhqKAADKg2AcABJEQxEAQBzBOAAkiIYiAIA4gnEASBANRQAAcQTjAJAgGooAyDR3adeucNnMOJaMYBwAEhY1FBkelrZtC5f791PWEEAGjI5K69dLF100E3i7h+vr14fb0VbUGQeQirI3vaGhCIBMGhiQNm0KqwSSNDQUAvHh4TA+MJDu/ArIvERvN/T19fnY2Fja0wBKr17Tm64umt4AQCZEK+FRQC6FQHxoSDJLb145Yma3u3tfU/clGAeQpEpF6u0Nl7V6ekK6BnnTAJAy99m7zaenCcRb0EowTs44gETR9AYAMi5aGY+L55CjrQjGkYhKRdqxQ9q6NVzWWxVFOdD0BgAyLJ6ismlTWCWJcsgJyDuCDZzouHr5wVu2kB9cVlHTm3oBOU1vACBlo6MzgXiUIz40FG4bHpb6+6Vzzkl3jgVDzjg6ivxg1OKcAIAMcw8B+cDA7BzxRuOoi5xxZAb5wahF0xsAyDCzsPJdG3A3GscZ9ZiNAAAgAElEQVSSkaaCjiI/GPVETW927gznwMqVoc44gTgAoGwIxjut5G/3kB/cGUVomEPTGwAASFPpvJK3lR0cnF2mNK6rK9yO1uzZE3KuN2+Wtm8Pl729YRwAAOQLwXinxdvKRgF5idrKkh/cXpVKqExTqcy82zA1NTM+OZnu/AAAQGtIU+m02pJAUWvZErWVJT+4fZrZEEvqBwAA+dHUyriZLTez06qfP8/Mujs7rYKJB+SRdgXi7tKuXXOL8DcaT0mUH3zppeGSQHxx2BALAECxLBiMm9m7JV0naUd16KWSru3kpAqnk21lS56TXjbRhth62BALAED+NLMy/gFJr5b0H5Lk7vdJemEnJ1UonW4rW/Kc9LJhQywAAMXSTDD+pLv/V3TFzA6QVPxE53Zp1FY2CqCXunJd+/26uub+PBQGG2IBoMByknqK9jJf4BdrZn8m6SeS3iXpfZLeL2nc3S/u/PTaq6+vz8fGxpL9oUnVGXefvWQ6PU0gXmCTk2yIBYDC2bUrpJjGF9Ti73iPjIQumMg8M7vd3fuauW8z1VT+QNKFku6VtEnSVyX9zeKnVzJR+9hmxxejUU46K+OFRcMcAFlXhOZkiYunnkrheZzU08KbNxivpqR81t3fKemvk5kSWlKbIx7/w5UIyAEAiduzJ/Q+mJ4OlZ66u6UtW0I63dq1ac8uwyiHXErNpKnskXSau/88mSl1TippKp3GW1oAgAypVEJX4Epl7m09PaHvBGl1CyD1NPdaSVNpZgPn/ZL+ycwuNrMPRB9LmyLaZmAgBNzxV8zRK+uREd7SAgAkqpnmZJhHJ8shI5OaCcZ/JOkGSQdLOjL2sWRmdqaZfd/MJsxsW53bt5jZ3Wb2XTO7ycxeGrvtGTO7o/pxXTvmk0tR7nntK+ZG4wAAdBDNyZag0+WQkUkLbuB09/8pSWb2/Or1/2zHD67mo18m6Y2S9kr6jpld5+53x+72L5L63P0JM3uvpO2SokrK/+nuq9sxFwAA0B5Rc7J6ATnNyRbQqByyFMb7+0k9LaBmOnC+3My+I2lc0riZfcvMjmvDzz5Z0oS7/6Bax/xqSWfH7+Dut7j7E9Wrt0la3oafCwAAOoTmZEtA6mkpNZOmcrmkP3T35e6+XNKHJP3fNvzsXkk/jl3fWx1rZKOkr8SuH2RmY2Z2m5lxdgIAkAE0J1sCUk9LqZk64z3ufkN0xd1vrDYCSoyZvV1Sn6T+2PBL3X2fmf2ipJvN7Hvufn+dr71QoU66jjnmmETmCwBAma1dG6qm0JwMWFgzwfgDZnaxpC9Ur79d0gNt+Nn7JB0du768OjaLmb1BYTW+392fisbdfV/18gdm9jVJJylUfpnF3S9XWN1XX18fOx8AAEgAzcmA5jSTpvJuhaB5t6R/UAia392Gn/0dSavMbIWZPVfSBkmzqqKY2UkK3T7PcvdHYuOHmtnzqp8fIelUSfGNnwAAAEDmNVNN5TFJ72v3D3b3p83s9yR9VdIBkq5w9381sw9LGnP36yR9QtIySV+2kCf1I3c/S9Jxkv7GzKYVXlB8rKYKC5Aq2kADAIBmNNOB8x8lbXD3n1WvHyrpb939NxOYX1sVsgMnmpZUgFyvDXRXF22gAQAoi1Y6cDaTM/6iKBCXJHf/qZm9ZNGzA1JQL0DesqX9AXKlEn5OvA10VGt33TraQAMAgNmayRmfNrNn63ubGSVJkCvxADkKjKemZsYnJ9v3s2gDDQAAWtFMMP7Hkr5hZp81s89J+rqkP+zorIA2SjJApg00AABoRTMbOP/BzE6WdIokl/QH8comQNYlGSDTBhoAALSi4cq4mR1tZodIkrv/RNLjkl4raYOZPSeh+QFLFgXI9bQ7QKYNNAAAaMV8aSpflnSIJJnZiZJ2SXpE0smSLuv81ID2SDJApg00AABoxXxpKge7+97q529XqAP+cTPrknRn56cGtEcUCDcqN9juAJk20AAAoFnzBeMW+/x1Ci3p5e7TZkZbeeRK0gEybaAXRmMkAADmD8b/PzP7O0kPSTpc0s2SZGYvlvTzBOYGtBUBcnYkVfcdAICsmy9n/AOSdkt6WNKvuft/VcdfIul/dnpiAIopybrvAABkXcOVcXeflvS3dcb/uaMzAlBozdR95x0MABLpbCiHBeuMA0A70RgJQDNIZ0NZNNOBEwDaJsm67wDyiXQ2lAnBOIBE0RgJwEKaSWcDimK+Dpw9Zva/zeyzZvaWmtv+ovNTA1BENEYCsBDS2VAm8+WMXyHpQUn/IOndZvZmSW93959LOjWJyQEoJhojAZhPlM5WLyAnnQ1FY+71+/eY2R3uvjp2/RJJb5B0lqSb3H1NMlNsn76+Ph8bG0t7GgAAYB6VitTbGy5r9fSEF/O8eEeWmdnt7t7XzH3nyxk/yMyevd3d/0TS5yR9XdJhS5ohAABAA6SzoUzmS1P5B0mvl3RDNODunzGzhyX9ZacnBgAAyot0NpRFwzSVIiJNBQAAAJ3WrjQVAAAAAB1EMA4AAACkZMFg3Mzm5JXXGwMAAAAS5y7t2hUumxnPmGZWxr/d5BgAAACQrNFRaf166aKLZgJv93B9/fpwe4Y1XOE2sxdKOkrS883seElWvekQSQcnMDcAAABgfgMD0qZN0vBwuD40FALx4eEwPjCQ7vwWMF+6yW9Kerek5ZIu00wwXpH0Pzs8LwAAAGSZe1h1HhiQzBYe7xSzEIBLIQCPgvJNm8J4EnNYggVLG5rZW9z9SwnNp6MobQgAANAmu3aFNJB40BulhwwPSyMj0jnnJDcfd6krloE9PZ1aIN7u0oYvNLNDqt/402b2bTN7/ZJmCAAAgHyLp4dE+dpppYdEPzsunkOeYc0E4xe6+3+Y2ekKOeS/K2l7Z6cFAACATIvSQ6KAvKtrJhBPMj2k9kXA9PTcFwkZ1kwwHj2CdZI+7+53Nvl1AAAAKLJ4vnYk6Tzt0dG5LwLiLxIyXk2lmaD6TjPbLelNkr5iZss0E6ADAACgrLKQHjIwEPLT4y8CooB8ZCTz1VSaCcbfJel/STrZ3Z+QdJCkjZ2cFAAAADIuK+khZmGjaO1qfKPxjFmwk6a7P2NmvyjpjZI+Kun5Ik0FAACg3Bqlh0hhvL8/2WoqOdVMacO/lPQcSa919+PM7DBJX3X3VyUxwXaitCEAAECbZKXOeAa1u7Tha9z9PZKelCR3f1zSc5cwP2SRe6gXWvvirNE4AAAot5ynh2RFM8H4z82sS9VNm2Z2uKTpjs4KyRsdDYX74zleUS7Y+vWZ34lcBJWKtGOHtHVruKxU0p4RAADotIY542Z2oLs/LekySf9P0pFm9ieS3iLpTxKaH5ISL9wvhZyvtAr3l9CePdK6dWHvy9SU1N0tbdki7d4trV2b9uwAAECnNMwZN7N/dvc11c9fIekNkkzSje5+V3JTbB9yxhcQ3xUdSbpwfwlVKlJvb/2V8J4eaf9+admyzs9h505pfFxatUoaHAw/GwAAtK6VnPH5gvF/cfeT2jqzlBGMN8E9dNCKTE8TiHfYjh3S5s1hRbxWd3d4bbSxg8VE663Kd3WxKg8AwGK1EozPV9rwSDPb0uhGd//zlmeGbGtUuJ+V8Y4aH68fiEthfGKicz+7UgmBeHxVPprLunXJrMoDAFBm823gPEDSMkk9DT5QJFkp3F9Cq1aF1eh6urullSs797N37gy/6nqmp8PtAPKBTeBoOyqtJWK+lfGH3P3Dic0E6cpQ4f6y5S8PDobNmvV0dYXbOyXNVXkA7cMmcHREVGktHhvEF+9GRmjq0wbzBePkJZTJwED4o4oX6I8C8v7+xKqplPEJpacnPL5GedudTBOJVuUb5at3clUeQHuQboaOodJaIubbwHlYtcFP53642ZmShhVSYna4+8dqbn+epM9LeqWkxyQNuvsD1dsulrRR0jOSPuDuX13o57GBM9uyUFUkTZOT4R2BiYkQBA8OJlNFpczHHCiCtDeBo+CotLYobenAmUAgfoBCDfPfkPRySeeb2ctr7rZR0k/dfaWkIUkfr37tyyVtkPQKSWdK+qvq90OOlT1/edmy8IR56aXhMokgOFqV7+mZyVvv7p4ZJxAHso90M3RUPG01QiDeVs104OyUkyVNuPsP3P2/JF0t6eya+5wt6crq59dIer2ZWXX8and/yt1/KGmi+v2QYzyhpGPt2rACPjwsbdsWLvfvL25aEFA0aW4CRwk0qrTG5s22mS9nvNN6Jf04dn2vpF9tdB93f9rM/l3S4dXx22q+trdzU0USyF9OT7Qq32ll25wLJCHNTeAouNpKa/GccYkV8jZJc2U8EWZ2oZmNmdnYo48+mvZ0MI/Bwdn9huJ4Qsm/PXtCfvrmzdL27eGytzeMA1g80s3QMY0qrUWbOkdH055hIaS5Mr5P0tGx68urY/Xus9fMDpT0CwobOZv5WkmSu18u6XIpbOBsy8zREWlWFUFnUe0B6Kwo3SzpTeAouIxUWiu6NIPx70haZWYrFALpDZLeWnOf6yRdIOmbks6VdLO7u5ldJ+nvzOzPJb1E0ipJ305s5ugYnlCKqZnNuVR7AJYmqXSzxLiHldd4IDjfONrPrH4d8UbjWJTUgvFqDvjvSfqqQmnDK9z9X83sw5LG3P06SZ+R9AUzm5D0uELArur9viTpbklPS3q/uz+TygNB2xXuCQVszgXQOhrOoCTSXBmXu++WtLtm7I9jnz8p6bwGX/tRSR/t6AQBtAWbcwG0jIYzKImGTX+KiKY/QDpoLgRgUWg4g5xqS9MfAGgXqj0AWBQazqAEUk1TAVAebM4F0LJGDWcIyFEgrIwDmOEu7do1t7Nao/EWRZtzL700XBKIA2iotuHM9PRMDjkdIFEgBOMAZkTVC+JPdNET4vr1NHgAkBwazqAkCMaRvA6vvmIJ4tULooCc6gUA0hA1nImnpEQBedSIBigAgnEkj9XX7KpdeerqmrsyBQBJiBrL1P7faTQO5BTBOJLH6mu2Ub0AAIDEEIyXQdbSQlh9zbZG1QtIHwKA7MnaczxaRjBeBllMC2H1NZuoXgAA+ZLF53i0hGC8DLKYFsLqazZRvQAA8iWLz/FoiXmJgp++vj4fGxtLexrpyFJL4dp/FENDc6+zQp4O9xBwDwzM/h00GgcApC9Lz/GQJJnZ7e7e19R9CcZLxD3kZ0emp9P5I921K7x1Fv9HEf9HMjISdsoDAIDmZOU5HpJaC8ZJUymLLKWFUDsWAID2ydJzPFpGMF4GWduUR+1YAADaI2vP8WjZgWlPAAlotClPCuP9/aSFAACQRzzH5x4542XApjwAAIqJ5/hMYgNnA6UNxgEAAJAYNnACnUCXMwAA0GYE40Cz6HIGAADajA2cQLPiXc6kuc2KKMkIAB1XqUg7d0rj49KqVdLgoNTTk/asgMUjZxxoBV3OACA1e/ZI69aF6n1TU1J3d+hzs3u3tHZt2rMDZrCBswGCcbQFXc4AIHGVitTbGy5r9fRI+/dLy5YlPy+gHjZwAp1ClzMASMXOnWHto57p6XA7kEcE40Cz6HKGHKhUpB07pK1bw2W9VUQgj8bHQ2pKPVNT0sREsvMB2oUNnECz6HKGjKuXT7tlC/m0KIZVq8I5XS8g7+6WVq5Mfk5AO5AzDjSLLmfIMPJpUXSc48gTcsaBTjALK9+1AXejcSBB5NOi6Hp6wrs8PT1hJVwKl9E4gTjyijQVACgA8mlRBmvXhhXwnTvDOb1yZagzTiCOPCMYB4AWZLXhCPm0mE9Wz9vFWLZM2rgx2Z9ZpOOH7CFnHCgxnmBak+WGI+TTopEsn7d5wPHDYtD0pwGCcWAGTzCtyUOwy+8UtfJw3mYZxw+LxQZOAPOqVELQVqnMpDVMTc2MT06mO78sysMGySifdnhY2rYtXO7fTyBeZnk4b7OM44ckEIwDJcQTTOvyskEyyqe99NJwyapdueXlvM2qwh0/d2nXrrlN6hqNIxEE40AJFe4JJgHRBsl62CCJrOK8XZrCHb/RUWn9+tldo6Pu0uvXh9uROIJxtI5X1rnX6SeYIrZkHxwM+df1dHWF24Gs4bxdmsIdv4GB0EV6eHgmIL/oopnu0gMDac+wlAjG0TpeWedeJ59g9uwJG542b5a2bw+Xvb1hPM9oOII84rxdmsIdPzNpaGgmIO/qmgnEh4ZoXpcSqqmgdbWvpIeG5l7nDzrzOlF5owyVByYnaTiC/OG8XZrCHT/32Ssy09M8b7cZpQ0bIBhvo3hAHiEQz512P8Hs2BFWwhs1nhkeTr5ZB1CL+vooNZ6/E9FKME4HTixO9FZX/I+ZP+TcaXcnOzaGIuvqvSO0ZQu12FES872zLfE8nhJyxrE40R90XDyHHKVUuMoDKBTq66P0RkfnppTGc8jZ85UKgnG0rvaV9fT03N3ZKKXCVR5AoVBfP9+KWKUpcQMD0sjI7BXwKCAfGaGaSkpIU0HrGr2ylsJ4f790zjnpzhGpiCoMNNoYmusNT8g90qjyi/SiNjGr//zcaByJSCUYN7PDJO2UdKykByS9xd1/WnOf1ZL+WtIhkp6R9FF331m97XOS+iX9e/Xuv+3udyQxd2jmlfXAwNxX1v39vLIuuagle6EqD6AQojSqRhuMSaPKpnh6UST6Ha5bV4wqTSi3tNJUtkm6yd1XSbqper3WE5Le6e6vkHSmpE+a2Qtit/++u6+ufhCIS8k144leQddu8mg0juYUqJkSLdmRRaRR5RPpRSi6tILxsyVdWf38SklzllLd/T53H69+vl/SI5KOTGyGeUQznnzj9wd0VOEauJQE6UUourRyxl/k7g9VP39Y0ovmu7OZnSzpuZLujw1/1Mz+WNWVdXd/qsHXXijpQkk65phjljrvbIu3uZXmNuMhfSTb+P0BHUcaVf6QXoSi61jTHzO7UdKL69z0IUlXuvsLYvf9qbsf2uD7HCXpa5IucPfbYmMPKwTol0u6390/vNCcStH0h2L+mbZgsxF+f0tGQxegWMrQ2RfFk/kOnGb2fUm/7u4PRcG2u/9ynfsdohCI/x93v6bB9/p1SR909zct9HNLEYxLtLnNqKbbz/P7W7SmjzGAXOFvG3nTSjCeVs74dZIuqH5+gaRra+9gZs+VtEvS52sD8WoALzMzhXzzuzo62zyhGU8mNd1shN/fotHQBSiuKL1oeFjati1c7t9PII5iSCsY/5ikN5rZuKQ3VK/LzPrMbEf1Pm+R9FpJv21md1Q/Vldv+6KZfU/S9yQdIekjyU4/o2jGk1lNVQPg97ckVFwAio0qTSiqVDZwuvtjkl5fZ3xM0u9UP/9bSX/b4Otf19EJ5hXNeDKrqWoA/P6WhIoLAIA8ogNnkdCMJ7OaqgbA729JqLgAAMijVDZwpqU0GzixeO5hhToeEM833iSqAXQexxgAkBV52MAJZFOHGu/QbKTzOMYAgDwiTQWI62DjHZqNdB7HGACQN6SpALVovAMAAJYg801/0kIwjqbReAcAACwSOePAUtB4BwAAJIRgHIij8Q4AAEgQGziBOBrvAACABBGMA3E03gEAAAkiGAfizOqvfDcaBwAAWAJyxgEAAICUEIwDAAAAKSEYBwAAAFJCMA4AAACkhGAcQH65S7t2za3/3mgcAPKO/3uFQzAOIL9GR6X162c3ZIoaN61fH24HgCLh/17hUNoQQH4NDMx0SJVCPfh4B1XqwgMoGv7vFY55id7O6Ovr87GxsbSnAaCdohWh6IlJmt1BFQCKhv97mWdmt7t7X1P3JRgHkHvuUlcs6256mickAMXG/71MayUYJ2ccQL5FK0Rx8VxKACga/u8VCsE4gPyKv1W7aVNYGYpyKRs8MVUq0o4d0tat4bJSSWHeALBYi/i/h2xjAyeA/BodnXlCinIlh4bCbcPDUn+/dM45z959zx5p3brw3DU1JXV3S1u2SLt3S2vXpvQYGqhUpJ07pfFxadUqaXBQ6ulJe1YAUtfi/z1kHznjAPLLPTwxDQzMzpWsM16pSL299VfCe3qk/fulZcsSmvcC6r1o6OrK5osGAAlr4f8e0kPOOJBlNGxoH7OwAlT7xFNnfOfOENzWMz0dbs+CSiUE4pVKCMSlcBmNT06mO780kFoExLTwfw/5QDCO9JUtOKVhQyrGx2eC21pTU9LERLLzaSQvLxqSsmdPeEdj82Zp+/Zw2dsbxhFTtv+jQIEQjCN9ZQtO4w0bosdMw4aOW7UqpHvU090trVyZ7HwaycuLhiTwLkELyvZ/FCgQgnGkr2zBabTZJnrMXV1zN+Og7QYHZ5fkjevqCrdnQV5eNCSBdwlaULb/o0CBsIET2VDGbmI0bEhcHjZG5mmjaadt3RpSUxrZtk269NLk5pN5Zfw/CmQUGziRP/HSTJEiP4HQsCEVa9eGYHZ4OARyw8PhelYCcSkE3Lt3h8tohby7e2a8LIG4xLsELSvb/1GgIAjGkQ1lCk5p2JCqZcukjRvDiurGjdkMbvPwoiEJeUktyowy/R8FCoRgHOkrW3DaqGFD9JjZaLU0BakqkYcXDZ3GuwQtKNv/UaBAyBlH+nbtCrv948Fp/IllZKRY3cRo2NBZZTufSmByMmzWnJgIqSmDgwTic3DeA5nSSs44wTjSR3CKdqpdIRwamnud8wlFw/9RIFMIxhsgGAdKgqoSAIAUEYw3QDAOlAilIwEAKaG0IYByo6oEACAnCMYBFAtVJQAAOXJg2hMAgLZqVDpSCuP9/VSVAABkBsE4gGIZGAhl3OLVI6KAvL8/jAMAkBEE4wCKxaz+ynejcQAAUkTOOAAAAJASgnEAAAAgJakE42Z2mJndYGbj1ctDG9zvGTO7o/pxXWx8hZl9y8wmzGynmT03udkDAAAA7ZHWyvg2STe5+ypJN1Wv1/Of7r66+nFWbPzjkobcfaWkn0ra2NnpAgAAAO2XVjB+tqQrq59fKanp8gZmZpJeJ+maxXw9AKB1lYq0Y4e0dWu4rFTSnhEAFENa1VRe5O4PVT9/WNKLGtzvIDMbk/S0pI+5+6ikwyX9zN2frt5nr6TeRj/IzC6UdKEkHXPMMe2YOwCUyp490rp1oX/S1JTU3S1t2SLt3i2tXZv27AAg3zoWjJvZjZJeXOemD8WvuLubWaOWeC91931m9ouSbjaz70n691bm4e6XS7pckvr6+mi9BwAtqFRCIB5fCZ+aCpfr1kn790vLlqUzN3RepSLt3CmNj0urVkmDg1JPT9qzKgH30MAs3i9hvnHkWsfSVNz9De7+K3U+rpX0EzM7SpKql480+B77qpc/kPQ1SSdJekzSC8wseiGxXNK+Tj0OACiznTvDing909PhdhTTnj1Sb6+0ebO0fXu47O0N4+iw0VFp/XrpootCAC6Fy4suCuOjo+nOD22VVs74dZIuqH5+gaRra+9gZoea2fOqnx8h6VRJd7u7S7pF0rnzfT0AYOnGx2dWwmtNTUkTE8nOB8mIvyMS/f6npmbGJyfTnV/hDQxImzZJw8MzAflFF4XrmzbRSbhg0soZ/5ikL5nZRkkPSnqLJJlZn6T/7u6/I+k4SX9jZtMKLxo+5u53V79+q6Srzewjkv5F0meSfgAAUAarVoUc8XoBeXe3tHJl8nNCY+1KK2nmHZGNGa5jlvv0GjNpaCh8PjwcPqQQiA8NkaJSMOZenjTqvr4+HxsbS3saAJAblUpITahXPaWnh5zxLKm30bara3EbbbduDakpjWzbJl166dLm2yntPA6pcw+Tj0xPE4jnhJnd7u59zdyXDpwAgIZ6ekIQ09MTghopXEbjBOLZ0O60kugdkXqy/I5IodJrotSUuHgOOQqDYBwAMK+1a8MK+PBwWBEdHg7Xc7fKWGDt3mg7ODh7QTauqyvcnkWF2XBcmyM+PT03hxyFkVbOOAAgR5Yty3aOcNm1e6Nt9M5Ho3SPrL4jkukNx62UKxwdnQnEoxzxeA55f790zjnJPwZ0BCvjAADkXCfSSvL4jkim02taKVc4MCCNjMzerBkF5CMjVFMpGDZwAgCQgE5W+GCjbZDp41CbejI0NPc6mzMLo5UNnKSpAADQYfUqfGzZ0r4KH3lNK2m3TB8HyhWiAVbGAQDooCRXaycnw+r7xERIyRgcLE8gHpfp40C5wlKgtCEAJMVd2rVrbnWDRuNFUdbHHWnh8SdZ4SPaaHvppeEyMwFowjJ7HChXiDoIxgFgKVrZlFUkZX3ckRYef6YrfCA5lCtEA+SMA8BSDAzMPKFKczdlFbXqQVkfd6SFxx9V+KgXkKde4QPJoVwhGiBnHACWKr7iFSnDpqyyPu5Ik48/0xU+4lqpg43WcXxLpZWccYJxAGiHsm7KWuLj7mS5v0Q0+fjrVVOJKnxkpm73rl0hxSb+giL+gmNkhJVboEls4ASAJJV1U9YSH/eePWHFePNmafv2cNnbG8ZzoYXHn4sGOvHUm+hxlCn1CEiLu5fm45WvfKUDQFtNT7tv2uQuhct614toiY/7P/7Dvacn3L32o6fHvVJJ6HEsVlF/7/HHEX3k+fEAKZE05k3Gp6yMA8BSNNqUFa0wFrWqyBIfd5Ll/jqiqL/3+KbCSFn2AAApoZoKACzFwEDIpY1vvooCmv7+4r61v8THnftyf0X9vTdKvSEgBzqGlXEAWAqzsKmtNlBpNF4US3zcUbm/enJR7q+Iv3fqYAOpIBgHACRucHB2EZK4rq5wOxJW1NQbIOMIxgEAievpCWX9enpmVsi7u2fGM1F3u2yi1Jt4SkoUkEcpOQDajjrjANqHphZo0eRk2Kw5MRFSUwYHCcQB5F8rdcbZwAmgfUZHaRqClixbJm3cmPYsACA9BOMA2ifeNEQKATlNQwAAaIhgHED7xGsUDw/PBOXxlXIAAPAscsYBtJ/77FIZ09ME4gCA0mglZ5xqKgDaq1HTkBK98AcAoFkE4wDah6YhAAC0hJxxAO3TqGmIFMb7+6mmAgBADME4gPaJmobE64lHAYFbWa4AAA1USURBVHl/P9VUAACoQTAOoH3M6q98NxoHAKDkyBkHAAAAUkIwDgAAAKSEYBwAAABICcE4AAAAkBKCcQAAACAlBOMAAABASgjGAQAAgJQQjAMAAAApIRgHAAAAUkIwDgAAAKSEYBwAAABICcE4AADoPHdp165w2cw4UBIE4wAAoPNGR6X166WLLpoJvN3D9fXrw+1ACR2Y9gQAAEAJDAxImzZJw8Ph+tBQCMSHh8P4wEC68wNSksrKuJkdZmY3mNl49fLQOvc5zczuiH08aWYD1ds+Z2Y/jN22OvlHAQA5QooA0mYWAvAoIO/qmgnEh4bC7UAJpZWmsk3STe6+StJN1euzuPst7r7a3VdLep2kJyRdH7vL70e3u/sdicwaAPKKFAFkQRSQxxGIo+TSCsbPlnRl9fMrJS303tS5kr7i7k90dFYAUFTxFIEoICdFAEmLzru4+AtEoITSCsZf5O4PVT9/WNKLFrj/BklX1Yx91My+a2ZDZva8Rl9oZhea2ZiZjT366KNLmDIA5FjZUwRI00lf7QvA6em5LxCBEjLv0MlvZjdKenGdmz4k6Up3f0Hsvj919zl549XbjpL0XUkvcfefx8YelvRcSZdLut/dP7zQnPr6+nxsbKzlxwIAheEeAvHI9HTxA3EpBNzr189+8REPDkdGpHPOSXuWxcbvACViZre7e18z9+1YNRV3f0Oj28zsJ2Z2lLs/VA2sH5nnW71F0q4oEK9+72hV/Skz+6ykD7Zl0gBQZI1SBMqwMk4lj/QNDISAe2Bg5nyL3rHp7+d3gNJKK03lOkkXVD+/QNK189z3fNWkqFQDeJmZKeSb39WBOQJAcZQ9RaDsaTpZYBZWvmuPdaNxoCQ6lqYy7w81O1zSlyQdI+lBSW9x98fNrE/Sf3f336ne71hJ35B0tLtPx77+ZklHSjJJd1S/ZnKhn0uaCoDSIkUgKGuaDoBEtZKmkkownhaCcQCl5R7KF8ZTBOYbL6L4i48IK+MAOqCVYDytNBUAQJLKniJQ9jQdAJnVsQ2cAABkxujo3BzxqPnM8HDYQFiGNB0AmUMwDgAoPip5AMgognEAQPFF6TjNjgNAQsgZBwAAAFJCMA4AAACkhGAcAAAASAnBOAAAAJASgnEAAAAgJQTjAAAAQEoIxgEAAICUEIwDAAAAKSEYBwAAAFJCMA4AAACkhGAcAAAASAnBOAAAAJASgnEAAAAgJQTjAAAAQEoIxgEAAICUEIwDAAAAKSEYBwAAAFJi7p72HBJjZo9KejDteSTgCEn/lvYkcozjt3gcu8Xj2C0Nx2/xOHZLw/FbvCIfu5e6+5HN3LFUwXhZmNmYu/elPY+84vgtHsdu8Th2S8PxWzyO3dJw/BaPYxeQpgIAAACkhGAcAAAASAnBeDFdnvYEco7jt3gcu8Xj2C0Nx2/xOHZLw/FbPI6dyBkHAAAAUsPKOAAAAJASgnEAAAAgJQTjOWVmh5nZDWY2Xr08tM59TjOzO2IfT5rZQPW2z5nZD2O3rU7+UaSjmWNXvd8zseNzXWx8hZl9y8wmzGynmT03udmnr8lzb7WZfdPM/tXMvmtmg7HbSnfumdmZZvb96jmzrc7tz6ueSxPVc+vY2G0XV8e/b2ZnJDnvLGji2G0xs7ur59lNZvbS2G11/4bLpInj99tm9mjsOP1O7LYLqn/n42Z2QbIzT18Tx24odtzuM7OfxW4r9blnZleY2SNmdleD283MPlU9tt81szWx28p33rk7Hzn8kLRd0rbq59skfXyB+x8m6XFJB1evf07SuWk/jiwfO0mTDca/JGlD9fNPS3pv2o8pa8dP0i9JWlX9/CWSHpL0gur1Up17kg6QdL+kX5T0XEl3Snp5zX3eJ+nT1c83SNpZ/fzl1fs/T9KK6vc5IO3HlLFjd1rs/9p7o2NXvV73b7gsH00ev9+W9Jd1vvYwST+oXh5a/fzQtB9Tlo5dzf3/h6QrYtfLfu69VtIaSXc1uH2dpK9IMkmvlvSt6ngpzztWxvPrbElXVj+/UtLAAvc/V9JX3P2Jjs4qH1o9ds8yM5P0OknXLObrC2LB4+fu97n7ePXz/ZIekdRUJ7ICOlnShLv/wN3/S9LVCscwLn5Mr5H0+uq5drakq939KXf/oaSJ6vcriwWPnbvfEvu/dpuk5QnPMcuaOfcaOUPSDe7+uLv/VNINks7s0DyzqNVjd76kqxKZWQ64+9cVFgAbOVvS5z24TdILzOwolfS8IxjPrxe5+0PVzx+W9KIF7r9Bc/9RfLT69tCQmT2v7TPMrmaP3UFmNmZmt0XpPZIOl/Qzd3+6en2vpN4OzjWLWjr3zOxkhZWl+2PDZTr3eiX9OHa93jnz7H2q59a/K5xrzXxtkbX6+DcqrLZF6v0Nl0mzx+/N1b/Ha8zs6Ba/tqiafvzV1KgVkm6ODZf93FtIo+NbyvPuwLQngMbM7EZJL65z04fiV9zdzaxhjcrqq83jJX01NnyxQiD1XIU6n1slfXipc86KNh27l7r7PjP7RUk3m9n3FIKkwmvzufcFSRe4+3R1uNDnHtJhZm+X1CepPzY852/Y3e+v/x1K6+8lXeXuT5nZexTeoXldynPKmw2SrnH3Z2JjnHtoGsF4hrn7GxrdZmY/MbOj3P2hasDzyDzf6i2Sdrn7z2PfO1rZfMrMPivpg22ZdEa049i5+77q5Q/M7GuSTpL0/xTeTjuwuoK5XNK+tj+AlLXj+JnZIZL+QdKHqm9DRt+70OdeHfskHR27Xu+cie6z18wOlPQLkh5r8muLrKnHb2ZvUHih2O/uT0XjDf6GyxQQLXj83P2x2NUdCntCoq/99Zqv/VrbZ5hdrfztbZD0/vgA596CGh3fUp53pKnk13WSol3GF0i6dp77zsllqwZRUQ70gKS6O54LasFjZ2aHRukTZnaEpFMl3e1hh8ktCjn4Db++4Jo5fs+VtEshJ/CamtvKdu59R9IqC1V4nqvwxF1bXSF+TM+VdHP1XLtO0gYL1VZWSFol6dsJzTsLFjx2ZnaSpL+RdJa7PxIbr/s3nNjMs6GZ43dU7OpZku6pfv5VSadXj+Ohkk7X7HdXi66Zv1vZ/9/e/Yb8NcZxHH9/Rv6EMJQ9WZq0KbQYaSGk1VrRkj8hbEqz5rEHssYUD5RSa2E2DzwgjyjK2qwoQ9LNzQqTNv+2Jfm3/Bn7enCu6ddsw6/l7Lf7/arT/fudc65znXN13e27677O9U1m0L1ouHFgn33vn70I3NpWVbkY+L4N1EzMftf3G6Ruw21080nXA58A64DJbf8sYNXAeWfQ/U9z0l7lXwXG6QKhZ4Dj+36mQ6ntgNmtfd5rP+8YKD+NLiDaDDwPHN33Mx2C7XcLsAsYG9hmTtS+R7dywMd0I2P3tn0P0AWQAMe0vrS59a1pA2XvbeU+Aub2/SyHYNutA7YP9LMX2/79/g5PpO1ftN9DwIetnTYAMwbKLmx9cjOwoO9nOdTarn1fBjy8V7kJ3/foBgC/bv8OfEH3PsciYFE7HmBFa9txYNZA2QnX79IeXJIkSdL/zGkqkiRJUk8MxiVJkqSeGIxLkiRJPTEYlyRJknpiMC5JkiT1xGBckkZAklOSjLVtW5IvB74f9R+uszDJvrKrkuSGJJuS7E4ycz/nHJFkRZIPkownebulA5ckDcEMnJI0AqrLlDgTIMky4KeqemSISy0E3gW27ePYOF0iptUHKH8T3Vrz51XV7iRTgR+GuI+/DGS0laQJx2BckkZcktvo0nEfBbwBLKH7y+caugA+wBN0yXFmAs8l+Rm4qKp+23OdqtrUrneg6qYAX1fV7lZm68B9zAOWA0cA26tqTstAuJouAdlPwJ1V9UGSB4GpwJnAZ0lup0vFfgldEqTHqmrV8K0iSaPBYFySRliSc4D5wOyq+j3JE3Spuz8FTq2qc9t5J1XVd0nuBpZU1diQVT4LvJ7kcrpMrM9U1Vib+rISuLSqtiSZ3M5fDrxVVVcnmQM8TZetFWAGcFlV/ZJkMbCjqi5qqcTfTLJ2MNiXpMORwbgkjbargAuBd9qI9rHA58ArwPQkjwEvAWsPRmVVtTXJdODKtm1IMh84GdhQVVvaed+2IpcA89q+tUmeTnJcO/ZCVf3SPs8Bzk5yY/t+InAWYDAu6bBmMC5Joy3A6qq6728HkvOAuXRTWK4F7jwYFbYA+mXg5STfANcArw1xqZ0DnwMsrqr1B+EWJWlkuJqKJI22dcD1bW72nlVXpiY5DUhVPQ8sBc5v5/8InDBsZUkuSDKlfZ4EnAtsoZurfsWelVUGpqm8Dtzc9l0FfFlVO/924W4kf3GSI9u505McO+x9StKocGRckkZYVY0nuR9Y14LjXcAi4A/gqXRzVwq4pxVZA6za1wucSa4DHgVOA15J8k5VzdurytOBJ9tyigE2Aiur6tckdwEvtDq/ohuVXwqsTvI+3QucC/bzKI/TvdA51qbb7KAbcZekw1qqqu97kCRJkiYkp6lIkiRJPTEYlyRJknpiMC5JkiT1xGBckiRJ6onBuCRJktQTg3FJkiSpJwbjkiRJUk/+BODlUaUbUGyUAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 864x576 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "positive = data2[data2['Accepted'].isin([1])]\n",
    "negative = data2[data2['Accepted'].isin([0])]\n",
    "\n",
    "fig, ax = plt.subplots(figsize=(12,8))\n",
    "ax.scatter(positive['Test 1'], positive['Test 2'], s=50, c='b', marker='o', label='Accepted')\n",
    "ax.scatter(negative['Test 1'], negative['Test 2'], s=50, c='r', marker='x', label='Rejected')\n",
    "ax.legend()\n",
    "ax.set_xlabel('Test 1 Score')\n",
    "ax.set_ylabel('Test 2 Score')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "哇，这个数据看起来可比前一次的复杂得多。特别地，你会注意到其中没有线性决策界限，来良好的分开两类数据。一个方法是用像逻辑回归这样的线性技术来构造从原始特征的多项式中得到的特征。让我们通过创建一组多项式特征入手吧。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Accepted</th>\n",
       "      <th>Ones</th>\n",
       "      <th>F10</th>\n",
       "      <th>F20</th>\n",
       "      <th>F21</th>\n",
       "      <th>F30</th>\n",
       "      <th>F31</th>\n",
       "      <th>F32</th>\n",
       "      <th>F40</th>\n",
       "      <th>F41</th>\n",
       "      <th>F42</th>\n",
       "      <th>F43</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0.051267</td>\n",
       "      <td>0.002628</td>\n",
       "      <td>0.035864</td>\n",
       "      <td>0.000135</td>\n",
       "      <td>0.001839</td>\n",
       "      <td>0.025089</td>\n",
       "      <td>0.000007</td>\n",
       "      <td>0.000094</td>\n",
       "      <td>0.001286</td>\n",
       "      <td>0.017551</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>-0.092742</td>\n",
       "      <td>0.008601</td>\n",
       "      <td>-0.063523</td>\n",
       "      <td>-0.000798</td>\n",
       "      <td>0.005891</td>\n",
       "      <td>-0.043509</td>\n",
       "      <td>0.000074</td>\n",
       "      <td>-0.000546</td>\n",
       "      <td>0.004035</td>\n",
       "      <td>-0.029801</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>-0.213710</td>\n",
       "      <td>0.045672</td>\n",
       "      <td>-0.147941</td>\n",
       "      <td>-0.009761</td>\n",
       "      <td>0.031616</td>\n",
       "      <td>-0.102412</td>\n",
       "      <td>0.002086</td>\n",
       "      <td>-0.006757</td>\n",
       "      <td>0.021886</td>\n",
       "      <td>-0.070895</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>-0.375000</td>\n",
       "      <td>0.140625</td>\n",
       "      <td>-0.188321</td>\n",
       "      <td>-0.052734</td>\n",
       "      <td>0.070620</td>\n",
       "      <td>-0.094573</td>\n",
       "      <td>0.019775</td>\n",
       "      <td>-0.026483</td>\n",
       "      <td>0.035465</td>\n",
       "      <td>-0.047494</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>-0.513250</td>\n",
       "      <td>0.263426</td>\n",
       "      <td>-0.238990</td>\n",
       "      <td>-0.135203</td>\n",
       "      <td>0.122661</td>\n",
       "      <td>-0.111283</td>\n",
       "      <td>0.069393</td>\n",
       "      <td>-0.062956</td>\n",
       "      <td>0.057116</td>\n",
       "      <td>-0.051818</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   Accepted  Ones       F10       F20       F21       F30       F31       F32  \\\n",
       "0         1     1  0.051267  0.002628  0.035864  0.000135  0.001839  0.025089   \n",
       "1         1     1 -0.092742  0.008601 -0.063523 -0.000798  0.005891 -0.043509   \n",
       "2         1     1 -0.213710  0.045672 -0.147941 -0.009761  0.031616 -0.102412   \n",
       "3         1     1 -0.375000  0.140625 -0.188321 -0.052734  0.070620 -0.094573   \n",
       "4         1     1 -0.513250  0.263426 -0.238990 -0.135203  0.122661 -0.111283   \n",
       "\n",
       "        F40       F41       F42       F43  \n",
       "0  0.000007  0.000094  0.001286  0.017551  \n",
       "1  0.000074 -0.000546  0.004035 -0.029801  \n",
       "2  0.002086 -0.006757  0.021886 -0.070895  \n",
       "3  0.019775 -0.026483  0.035465 -0.047494  \n",
       "4  0.069393 -0.062956  0.057116 -0.051818  "
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "degree = 5\n",
    "x1 = data2['Test 1']\n",
    "x2 = data2['Test 2']\n",
    "\n",
    "data2.insert(3, 'Ones', 1)\n",
    "\n",
    "for i in range(1, degree):\n",
    "    for j in range(0, i):\n",
    "        data2['F' + str(i) + str(j)] = np.power(x1, i-j) * np.power(x2, j)\n",
    "\n",
    "data2.drop('Test 1', axis=1, inplace=True)\n",
    "data2.drop('Test 2', axis=1, inplace=True)\n",
    "\n",
    "data2.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "现在，我们需要修改第1部分的成本和梯度函数，包括正则化项。首先是成本函数："
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# regularized cost（正则化代价函数）\n",
    "$$J\\left( \\theta  \\right)=\\frac{1}{m}\\sum\\limits_{i=1}^{m}{[-{{y}^{(i)}}\\log \\left( {{h}_{\\theta }}\\left( {{x}^{(i)}} \\right) \\right)-\\left( 1-{{y}^{(i)}} \\right)\\log \\left( 1-{{h}_{\\theta }}\\left( {{x}^{(i)}} \\right) \\right)]}+\\frac{\\lambda }{2m}\\sum\\limits_{j=1}^{n}{\\theta _{j}^{2}}$$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [],
   "source": [
    "def costReg(theta, X, y, learningRate):\n",
    "    theta = np.matrix(theta)\n",
    "    X = np.matrix(X)\n",
    "    y = np.matrix(y)\n",
    "    first = np.multiply(-y, np.log(sigmoid(X * theta.T)))\n",
    "    second = np.multiply((1 - y), np.log(1 - sigmoid(X * theta.T)))\n",
    "    reg = (learningRate / (2 * len(X))) * np.sum(np.power(theta[:,1:theta.shape[1]], 2))\n",
    "    return np.sum(first - second) / len(X) + reg"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "请注意等式中的\"reg\" 项。还注意到另外的一个“学习率”参数。这是一种超参数，用来控制正则化项。现在我们需要添加正则化梯度函数："
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "如果我们要使用梯度下降法令这个代价函数最小化，因为我们未对${{\\theta }_{0}}$ 进行正则化，所以梯度下降算法将分两种情形：\n",
    "\\begin{align}\n",
    "  & Repeat\\text{ }until\\text{ }convergence\\text{ }\\!\\!\\{\\!\\!\\text{ } \\\\ \n",
    " & \\text{     }{{\\theta }_{0}}:={{\\theta }_{0}}-a\\frac{1}{m}\\sum\\limits_{i=1}^{m}{[{{h}_{\\theta }}\\left( {{x}^{(i)}} \\right)-{{y}^{(i)}}]x_{_{0}}^{(i)}} \\\\ \n",
    " & \\text{     }{{\\theta }_{j}}:={{\\theta }_{j}}-a\\frac{1}{m}\\sum\\limits_{i=1}^{m}{[{{h}_{\\theta }}\\left( {{x}^{(i)}} \\right)-{{y}^{(i)}}]x_{j}^{(i)}}+\\frac{\\lambda }{m}{{\\theta }_{j}} \\\\ \n",
    " & \\text{          }\\!\\!\\}\\!\\!\\text{ } \\\\ \n",
    " & Repeat \\\\ \n",
    "\\end{align}\n",
    "\n",
    "对上面的算法中 j=1,2,...,n 时的更新式子进行调整可得： \n",
    "${{\\theta }_{j}}:={{\\theta }_{j}}(1-a\\frac{\\lambda }{m})-a\\frac{1}{m}\\sum\\limits_{i=1}^{m}{({{h}_{\\theta }}\\left( {{x}^{(i)}} \\right)-{{y}^{(i)}})x_{j}^{(i)}}$\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [],
   "source": [
    "def gradientReg(theta, X, y, learningRate):\n",
    "    theta = np.matrix(theta)\n",
    "    X = np.matrix(X)\n",
    "    y = np.matrix(y)\n",
    "    \n",
    "    parameters = int(theta.ravel().shape[1])\n",
    "    grad = np.zeros(parameters)\n",
    "    \n",
    "    error = sigmoid(X * theta.T) - y\n",
    "    \n",
    "    for i in range(parameters):\n",
    "        term = np.multiply(error, X[:,i])\n",
    "        \n",
    "        if (i == 0):\n",
    "            grad[i] = np.sum(term) / len(X)\n",
    "        else:\n",
    "            grad[i] = (np.sum(term) / len(X)) + ((learningRate / len(X)) * theta[:,i])\n",
    "    \n",
    "    return grad"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "就像在第一部分中做的一样，初始化变量。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [],
   "source": [
    "# set X and y (remember from above that we moved the label to column 0)\n",
    "cols = data2.shape[1]\n",
    "X2 = data2.iloc[:,1:cols]\n",
    "y2 = data2.iloc[:,0:1]\n",
    "\n",
    "# convert to numpy arrays and initalize the parameter array theta\n",
    "X2 = np.array(X2.values)\n",
    "y2 = np.array(y2.values)\n",
    "theta2 = np.zeros(11)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "让我们初始学习率到一个合理值。，果有必要的话（即如果惩罚太强或不够强）,我们可以之后再折腾这个。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [],
   "source": [
    "learningRate = 1"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "现在，让我们尝试调用新的默认为0的theta的正则化函数，以确保计算工作正常。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.6931471805599454"
      ]
     },
     "execution_count": 24,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "costReg(theta2, X2, y2, learningRate)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([0.00847458, 0.01878809, 0.05034464, 0.01150133, 0.01835599,\n",
       "       0.00732393, 0.00819244, 0.03934862, 0.00223924, 0.01286005,\n",
       "       0.00309594])"
      ]
     },
     "execution_count": 25,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "gradientReg(theta2, X2, y2, learningRate)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "现在我们可以使用和第一部分相同的优化函数来计算优化后的结果。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(array([ 0.53010249,  0.29075567, -1.60725763, -0.5821382 ,  0.01781027,\n",
       "        -0.21329509, -0.40024142, -1.37144139,  0.02264303, -0.9503358 ,\n",
       "         0.0344085 ]), 22, 1)"
      ]
     },
     "execution_count": 26,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "result2 = opt.fmin_tnc(func=costReg, x0=theta2, fprime=gradientReg, args=(X2, y2, learningRate))\n",
    "result2"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "最后，我们可以使用第1部分中的预测函数来查看我们的方案在训练数据上的准确度。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "accuracy = 0.6610169491525424%\n"
     ]
    }
   ],
   "source": [
    "theta_min = np.matrix(result2[0])\n",
    "predictions = predict(theta_min, X2)\n",
    "correct = [1 if ((a == 1 and b == 1) or (a == 0 and b == 0)) else 0 for (a, b) in zip(predictions, y2)]\n",
    "accuracy = (sum(map(int, correct)) / len(correct))\n",
    "print ('accuracy = {0}%'.format(accuracy))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "虽然我们实现了这些算法，值得注意的是，我们还可以使用高级Python库像scikit-learn来解决这个问题。"
   ]
  },
  {
   "cell_type": "code",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2024-08-07T02:06:18.256669Z",
     "start_time": "2024-08-07T02:06:18.228910Z"
    }
   },
   "source": [
    "from sklearn import linear_model#调用sklearn的线性回归包\n",
    "model = linear_model.LogisticRegression(penalty='l2', C=1.0)\n",
    "model.fit(X2, y2.ravel())"
   ],
   "outputs": [
    {
     "ename": "ModuleNotFoundError",
     "evalue": "No module named 'sklearn'",
     "output_type": "error",
     "traceback": [
      "\u001B[1;31m---------------------------------------------------------------------------\u001B[0m",
      "\u001B[1;31mModuleNotFoundError\u001B[0m                       Traceback (most recent call last)",
      "Cell \u001B[1;32mIn[14], line 1\u001B[0m\n\u001B[1;32m----> 1\u001B[0m \u001B[38;5;28;01mfrom\u001B[39;00m \u001B[38;5;21;01msklearn\u001B[39;00m \u001B[38;5;28;01mimport\u001B[39;00m linear_model\u001B[38;5;66;03m#调用sklearn的线性回归包\u001B[39;00m\n\u001B[0;32m      2\u001B[0m model \u001B[38;5;241m=\u001B[39m linear_model\u001B[38;5;241m.\u001B[39mLogisticRegression(penalty\u001B[38;5;241m=\u001B[39m\u001B[38;5;124m'\u001B[39m\u001B[38;5;124ml2\u001B[39m\u001B[38;5;124m'\u001B[39m, C\u001B[38;5;241m=\u001B[39m\u001B[38;5;241m1.0\u001B[39m)\n\u001B[0;32m      3\u001B[0m model\u001B[38;5;241m.\u001B[39mfit(X2, y2\u001B[38;5;241m.\u001B[39mravel())\n",
      "\u001B[1;31mModuleNotFoundError\u001B[0m: No module named 'sklearn'"
     ]
    }
   ],
   "execution_count": 14
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.6610169491525424"
      ]
     },
     "execution_count": 29,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model.score(X2, y2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "这就是练习2的全部！ 敬请期待下一个练习：多类图像分类。"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.8"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}
